elasticsearch/docs/reference/search/async-search.asciidoc

[role="xpack"]
[testenv="basic"]
[[async-search]]
=== Async search

The async search API let you asynchronously execute a
search request, monitor its progress, and retrieve  partial results
as they become available.

[[submit-async-search]]
==== Submit async search API

Executes a search request asynchronously. It accepts the same
parameters and request body as the <<search-search,search API>>.

[source,console,id=submit-async-search-date-histogram-example]
--------------------------------------------------
POST /sales*/_async_search?size=0
{
    "sort" : [
      { "date" : {"order" : "asc"} }
    ],
    "aggs" : {
        "sale_date" : {
             "date_histogram" : {
                 "field" : "date",
                 "calendar_interval": "1d"
             }
         }
    }
}
--------------------------------------------------
// TEST[setup:sales]
// TEST[s/size=0/size=0&wait_for_completion=0/]

The response contains an identifier of the search being executed.
You can use this ID to later retrieve the search's final results.
The currently available search
results are returned as part of the <<search-api-response-body,`response`>> object.

[source,console-result]
--------------------------------------------------
{
  "id" : "FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=", <1>
  "version" : 0,
  "is_partial" : true, <2>
  "is_running" : true, <3>
  "start_time_in_millis" : 1583945890986,
  "expiration_time_in_millis" : 1584377890986,
  "response" : {
    "took" : 1122,
    "timed_out" : false,
    "num_reduce_phases" : 0,
    "_shards" : {
      "total" : 562, <4>
      "successful" : 3, <5>
      "skipped" : 0,
      "failed" : 0
    },
    "hits" : {
      "total" : {
        "value" : 157483, <6>
        "relation" : "gte"
      },
      "max_score" : null,
      "hits" : [ ]
    }
  }
}
--------------------------------------------------
// TESTRESPONSE[s/FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=/$body.id/]
// TESTRESPONSE[s/1583945890986/$body.start_time_in_millis/]
// TESTRESPONSE[s/1584377890986/$body.expiration_time_in_millis/]
// TESTRESPONSE[s/"took" : 1122/"took": $body.response.took/]
// TESTRESPONSE[s/"total" : 562/"total": $body.response._shards.total/]
// TESTRESPONSE[s/"successful" : 3/"successful": $body.response._shards.successful/]
// TESTRESPONSE[s/"value" : 157483/"value": $body.response.hits.total.value/]

<1> Identifier of the async search that can be used to monitor its progress, retrieve its results, and/or delete it.
<2> Whether the returned search results are partial or final
<3> Whether the search is still being executed or it has completed
<4> How many shards the search will be executed on, overall
<5> How many shards have successfully completed the search
<6> How many documents are currently matching the query, which belong to the shards that have already completed the search

It is possible to block and wait until the search is completed up to a certain
timeout by providing the `wait_for_completion` parameter, which defaults to
`1` second.

You can also specify how long the async search needs to be
available through the `keep_alive` parameter, which defaults to `5d` (five days).
Ongoing async searches and any saved search results are deleted after this
period.

NOTE: When results are sorted by a numeric field, shards get sorted based on
minimum and maximum value that they hold for that field, hence partial
results become available following the sort criteria that was requested.

The submit async search API supports the same <<search-search-api-query-params,parameters>>
as the search API, though some have different default values:

* `batched_reduce_size` defaults to `5`: this affects how often partial results
become available, which happens whenever shard results are reduced. A partial
reduction is performed every time the coordinating node has received a certain
number of new shard responses (`5` by default).
* `request_cache` defaults to `true`
* `pre_filter_shard_size` defaults to `1`: this is to enforce the execution of
a pre-filter roundtrip to retrieve statistics from each shard so that the ones
that surely don't hold any document matching the query get skipped.
* `ccs_minimize_roundtrips` defaults to `false`, which is also the only
supported value

WARNING: Async search does not support <<request-body-search-scroll,scroll>>
nor search requests that only include the  <<search-suggesters,suggest section>>.
{ccs} is supported only with <<ccs-min-roundtrips,`ccs_minimize_roundtrips`>>
set to `false`.

[[get-async-search]]
==== Get async search

The get async search API retrieves the results of a previously submitted
async search request given its id. If the {es} {security-features} are enabled.
the access to the results of a specific async search is restricted to the user
that submitted it in the first place.

[source,console,id=get-async-search-date-histogram-example]
--------------------------------------------------
GET /_async_search/FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=
--------------------------------------------------
// TEST[continued s/FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=/\${body.id}/]

[source,console-result]
--------------------------------------------------
{
  "id" : "FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=",
  "version" : 2, <1>
  "is_partial" : true, <2>
  "is_running" : true, <3>
  "start_time_in_millis" : 1583945890986,
  "expiration_time_in_millis" : 1584377890986, <4>
  "response" : {
    "took" : 12144,
    "timed_out" : false,
    "num_reduce_phases" : 38,
    "_shards" : {
      "total" : 562,
      "successful" : 188,
      "skipped" : 0,
      "failed" : 0
    },
    "hits" : {
      "total" : {
        "value" : 456433,
        "relation" : "eq"
      },
      "max_score" : null,
      "hits" : [ ]
    },
    "aggregations" : { <5>
      "sale_date" :  {
        "buckets" : []
      }
    }
  }
}
--------------------------------------------------
// TESTRESPONSE[s/FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=/$body.id/]
// TESTRESPONSE[s/"is_partial" : true/"is_partial" : false/]
// TESTRESPONSE[s/"is_running" : true/"is_running" : false/]
// TESTRESPONSE[s/1583945890986/$body.start_time_in_millis/]
// TESTRESPONSE[s/1584377890986/$body.expiration_time_in_millis/]
// TESTRESPONSE[s/"took" : 12144/"took": $body.response.took/]
// TESTRESPONSE[s/"total" : 562/"total": $body.response._shards.total/]
// TESTRESPONSE[s/"successful" : 188/"successful": $body.response._shards.successful/]
// TESTRESPONSE[s/"value" : 456433/"value": $body.response.hits.total.value/]
// TESTRESPONSE[s/"buckets" : \[\]/"buckets": $body.response.aggregations.sale_date.buckets/]
// TESTRESPONSE[s/"num_reduce_phases" : 38,//]

<1> The returned `version` is useful to identify whether the response contains
additional results compared to previously obtained responses. If the version
stays the same, no new results have become available, otherwise a higher version
number indicates that more shards have completed their execution of the query
and their partial results are also included in the response.
<2> Whether the returned search results are partial or final
<3> Whether the search is still being executed or it has completed
<4> When the async search will expire
<5> Partial aggregations results, coming from the shards that have already
completed the execution of the query.

The `wait_for_completion` parameter, which defaults to `1`, can also be provided
when calling the Get Async Search API, in order to wait for the search to be
completed up until the provided timeout. Final results will be returned if
available before the timeout expires, otherwise the currently available results
will be returned once the timeout expires.

The `keep_alive` parameter specifies how long the async search should be
available in the cluster. When not specified, the `keep_alive` set with the
corresponding submit async request will be used. Otherwise, it is possible to
override such value and extend the validity of the request. When this period
expires, the search, if still running, is cancelled. If the search is
completed, its saved results are deleted.

[[delete-async-search]]
==== Delete async search

You can use the delete async search API to manually delete an async search
by ID. If the search is still running, the search request will be cancelled.
Otherwise, the saved search results are deleted.

[source,console,id=delete-async-search-date-histogram-example]
--------------------------------------------------
DELETE /_async_search/FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=
--------------------------------------------------
// TEST[continued s/FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=/\${body.id}/]