Add ES|QL async query api docs (#104054)

This commit adds detailed API docs for the ES|QL async apis, as well as moving the narrative to the ES|QL specific REST docs. Additionally, a few top-level lists are reflowed to reflect that there are now multiple APIs.

docs/reference/esql/esql-apis.asciidoc

@@ -0,0 +1,20 @@
+[[esql-apis]]
+== {esql} APIs
+
+The {es} Query Language ({esql}) provides a powerful way to filter, transform,
+and analyze data stored in {es}, and in the future in other runtimes. For an
+overview of {esql} and related tutorials, see <<esql>>.
+
+* <<esql-query-api>>
+* <<esql-async-query-api>>
+* <<esql-async-query-get-api>>
+* <<esql-async-query-delete-api>>
+
+
+include::esql-query-api.asciidoc[]
+
+include::esql-async-query-api.asciidoc[]
+
+include::esql-async-query-get-api.asciidoc[]
+
+include::esql-async-query-delete-api.asciidoc[]

docs/reference/esql/esql-async-query-api.asciidoc

@@ -1,27 +1,17 @@
 [[esql-async-query-api]]
-== {esql} async query API
+=== {esql} async query API
 ++++
 <titleabbrev>{esql} async query API</titleabbrev>
 ++++
 
-Runs an async {esql} search.
+Runs an async <<esql,{esql} query>>.
 
-The async query API lets you asynchronously execute a search request,
-monitor its progress, and retrieve results as they become available.
+The async query API lets you asynchronously execute a query request,
+monitor its progress, and retrieve results when they become available.
 
-Executing an <<esql,ES|QL ({es} query language)>> is commonly quite fast,
-however searches across large data sets or frozen data can take some time.
-To avoid long waits, run an async {esql} search.
-
-Searches initiated by this API may return search results or not. The
-`wait_for_completion_timeout` property determines how long to wait for
-the search results. The default value is 1 second. If the results are
-not available by this time, a search id is return which can be later
-used to retrieve the results.
-
-Initiates an async search for an <<esql,ES|QL ({es} query language)>>
-query. The API accepts the same parameters and request body as the
-<<esql-query-api,query API>>.
+The API accepts the same parameters and request body as the synchronous
+<<esql-query-api,query API>>, along with additional async related
+properties as outline below.
 
 [source,console]
 ----
@@ -39,12 +29,12 @@ POST /_query/async
 ----
 // TEST[setup:library]
 
-If the results are not available within the timeout period, 2 seconds in
-this case, the search returns no results but rather a response that
+If the results are not available within the given timeout period, 2 seconds
+in this case, no results are returned but rather a response that
 includes:
 
- * A search ID
- * An `is_running` value of true, indicating the search is ongoing
+ * A query ID
+ * An `is_running` value of _true_, indicating the query is ongoing
 
 The query continues to run in the background without blocking other
 requests.
@@ -56,37 +46,119 @@ requests.
   "is_running": true
 }
 ----
-// TEST[skip: no access to search ID - may return response values]
-
-To check the progress of an async search, use the <<get-async-esql-query-api,get
-async ES|QL query API>> with the search ID. Specify how long you'd like for
-complete results in the `wait_for_completion_timeout` parameter.
-
-[source,console]
-----
-GET /_query/async/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=?wait_for_completion_timeout=30s
-----
-// TEST[skip: no access to search ID - may return response values]
+// TEST[skip: no access to query ID - may return response values]
 
-If the response's `is_running` value is `false`, the async search has
-finished, and the results are returned.
+Otherwise, if the response's `is_running` value is `false`, the async
+query has  finished, and the results are returned.
 
 [source,console-result]
 ----
 {
-  "id": "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
   "is_running": false,
   "columns": ...
 }
 ----
-// TEST[skip: no access to search ID - may return response values]
-
-Use the <<delete-async-eqsl-query-api,delete async ES|QL query API>> to
-delete an async search before the `keep_alive` period ends. If the query
-is still running, {es} cancels it.
-
-[source,console]
-----
-DELETE /_query/async/FmdMX2pIang3UWhLRU5QS0lqdlppYncaMUpYQ05oSkpTc3kwZ21EdC1tbFJXQToxOTI=
-----
-// TEST[skip: no access to search ID]
+// TEST[skip: no access to query ID - may return response values]
+
+[[esql-async-query-api-request]]
+==== {api-request-title}
+
+`POST /_query/async`
+
+[[esql-async-query-api-prereqs]]
+==== {api-prereq-title}
+
+* If the {es} {security-features} are enabled, you must have the `read`
+<<privileges-list-indices,index privilege>> for the data stream, index,
+or alias you query.
+
+[[esql-async-query-api-path-params]]
+==== {api-path-parms-title}
+
+The API accepts the same parameters as the synchronous
+<<esql-query-api-query-params,query API>>.
+
+[[esql-async-query-api-request-body]]
+==== {api-request-body-title}
+
+The API accepts the same request body as the synchronous
+<<esql-query-api-request-body,query API>>, along with the following
+parameters:
+
+[[esql-async-query-api-wait-for-completion-timeout]]
+`wait_for_completion_timeout`::
++
+--
+(Optional, <<time-units,time value>>)
+Timeout duration to wait for the request to finish. Defaults to a 1 second,
+meaning the request waits for 1 second for the query results.
+
+If this parameter is specified and the request completes during this period,
+complete results are returned.
+
+If the request does not complete during this period, a query
+<<esql-async-query-api-response-body-query-id,id>> is returned.
+--
+
+[[esql-async-query-api-keep-on-completion]]
+`keep_on_completion`::
++
+--
+(Optional, Boolean)
+If `true`, the query and its results are stored in the cluster.
+
+If `false`, the query and its results are stored in the cluster only if the
+request does not complete during the period set by the
+<<esql-async-query-api-wait-for-completion-timeout,`wait_for_completion_timeout`>>
+parameter. Defaults to `false`.
+--
+
+`keep_alive`::
++
+--
+(Optional, <<time-units,time value>>)
+Period for which the query and its results are stored in the cluster. Defaults
+to `5d` (five days).
+
+When this period expires, the query and its results are deleted, even if the
+query is still ongoing.
+
+If the <<esql-async-query-api-keep-on-completion,`keep_on_completion`>> parameter
+is `false`, {es} only stores async queries that do not complete within the period
+set by the <<esql-async-query-api-wait-for-completion-timeout,`wait_for_completion_timeout`>>
+parameter, regardless of this value.
+--
+
+[[esql-async-query-api-response-body]]
+==== {api-response-body-title}
+
+The API returns the same response body as the synchronous
+<<esql-query-api-response-body,query API>>, along with the following
+properties:
+
+[[esql-async-query-api-response-body-query-id]]
+`id`::
++
+--
+(string)
+Identifier for the query.
+
+This query ID is only provided if one of the following conditions is met:
+
+* A query request does not return complete results during the
+<<esql-async-query-api-wait-for-completion-timeout,`wait_for_completion_timeout`>>
+parameter's timeout period.
+
+* The query request's <<esql-async-query-api-keep-on-completion,`keep_on_completion`>>
+parameter is `true`.
+
+You can use this ID with the <<esql-async-query-get-api,{esql} async query get
+API>> to get the current status and available results for the query.
+--
+
+`is_running`::
++
+--
+(Boolean)
+If `true`, the query request is still executing.
+--

docs/reference/esql/esql-async-query-delete-api.asciidoc

@@ -0,0 +1,42 @@
+[[esql-async-query-delete-api]]
+=== {esql} async query delete API
+++++
+<titleabbrev>{esql} async query delete API</titleabbrev>
+++++
+
+The {esql} async query delete API is used to manually delete an async query
+by ID. If the query is still running, the query will be cancelled. Otherwise,
+the stored results are deleted.
+
+[source,console]
+----
+DELETE /query/async/FkpMRkJGS1gzVDRlM3g4ZzMyRGlLbkEaTXlJZHdNT09TU2VTZVBoNDM3cFZMUToxMDM=
+----
+// TEST[skip: no access to query ID]
+
+[[esql-async-query-delete-api-request]]
+==== {api-request-title}
+
+`DELETE /_query/async/<query_id>`
+
+[[esql-async-query-delete-api-prereqs]]
+==== {api-prereq-title}
+
+* If the {es} {security-features} are enabled, only the following users can
+use this API to delete a query:
+
+** The authenticated user that submitted the original query request
+** Users with the `cancel_task` <<privileges-list-cluster,cluster privilege>>
+
+
+[[esql-async-query-delete-api-path-params]]
+==== {api-path-parms-title}
+
+`<query_id>`::
+(Required, string)
+Identifier for the query to delete.
++
+A query ID is provided in the <<esql-async-query-api,{esql} async query API>>'s
+response for a query that does not complete in the awaited time. A query ID is
+also provided if the request's <<esql-async-query-api-keep-on-completion,`keep_on_completion`>>
+parameter is `true`.

docs/reference/esql/esql-async-query-get-api.asciidoc

@@ -0,0 +1,58 @@
+[[esql-async-query-get-api]]
+=== {esql} async query get API
+++++
+<titleabbrev>{esql} async query get API</titleabbrev>
+++++
+
+Returns the current status and available results for an <<esql-async-query-api,{esql}
+async query>> or a stored results.
+
+[source,console]
+----
+GET /_query/async/FkpMRkJGS1gzVDRlM3g4ZzMyRGlLbkEaTXlJZHdNT09TU2VTZVBoNDM3cFZMUToxMDM=
+----
+// TEST[skip: no access to query ID]
+
+[[esql-async-query-get-api-request]]
+==== {api-request-title}
+
+`GET /_query/async/<query_id>`
+
+[[esql-async-query-get-api-prereqs]]
+==== {api-prereq-title}
+
+* If the {es} {security-features} are enabled, only the user who first submitted
+the {esql} query can retrieve the results using this API.
+
+[[esql-async-query-get-api-path-params]]
+==== {api-path-parms-title}
+
+`<query_id>`::
+(Required, string)
+Identifier for the query.
++
+A query ID is provided in the <<esql-async-query-api,{esql} async query API>>'s
+response for a query that does not complete in the awaited time. A query ID is
+also provided if the request's <<esql-async-query-api-keep-on-completion,`keep_on_completion`>>
+parameter is `true`.
+
+[[esql-async-query-get-api-query-params]]
+==== {api-query-parms-title}
+
+`wait_for_completion_timeout`::
+(Optional, <<time-units,time value>>)
+Timeout duration to wait for the request to finish. Defaults to no timeout,
+meaning the request waits for complete query results.
++
+If this parameter is specified and the request completes during this period,
+complete query results are returned.
++
+If the request does not complete during this period, the response returns an
+`is_running` value of `true` and no results.
+
+[[esql-async-query-get-api-response-body]]
+==== {api-response-body-title}
+
+The {esql} async query get API returns the same response body as the {esql}
+query API. See the {esql} query API's <<esql-query-api-response-body,response
+body parameters>>.

docs/reference/esql/esql-query-api.asciidoc

@@ -1,5 +1,5 @@
 [[esql-query-api]]
-== {esql} query API
+=== {esql} query API
 ++++
 <titleabbrev>{esql} query API</titleabbrev>
 ++++
@@ -23,13 +23,13 @@ POST /_query
 
 [discrete]
 [[esql-query-api-request]]
-=== {api-request-title}
+==== {api-request-title}
 
 `POST _query`
 
 [discrete]
 [[esql-query-api-prereqs]]
-=== {api-prereq-title}
+==== {api-prereq-title}
 
 * If the {es} {security-features} are enabled, you must have the `read`
 <<privileges-list-indices,index privilege>> for the data stream, index,
@@ -37,7 +37,7 @@ or alias you search.
 
 [discrete]
 [[esql-query-api-query-params]]
-=== {api-query-parms-title}
+==== {api-query-parms-title}
 
 `delimiter`::
 (Optional, string) Separator for CSV results. Defaults to `,`. The API only
@@ -54,7 +54,7 @@ precedence.
 [discrete]
 [role="child_attributes"]
 [[esql-query-api-request-body]]
-=== {api-request-body-title}
+==== {api-request-body-title}
 
 `columnar`::
 (Optional, Boolean) If `true`, returns results in a columnar format. Defaults to
@@ -71,7 +71,7 @@ responses. See <<esql-rest-columnar>>.
 [discrete]
 [role="child_attributes"]
 [[esql-query-api-response-body]]
-=== {api-response-body-title}
+==== {api-response-body-title}
 
 `columns`::
 (array of objects)
@@ -79,13 +79,13 @@ Column headings for the search results. Each object is a column.
 +
 .Properties of `columns` objects
 [%collapsible%open]
-====
+=====
 `name`::
 (string) Name of the column.
 
 `type`::
 (string) Data type for the column.
-====
+=====
 
 `rows`::
 (array of arrays)

docs/reference/esql/esql-rest.asciidoc

@@ -247,3 +247,89 @@ POST /_query
 }
 ----
 // TEST[setup:library]
+
+[discrete]
+[[esql-rest-async-query]]
+==== Running an async {esql} query
+
+The <<esql-async-query-api,{esql} async query API>> lets you asynchronously
+execute a query request, monitor its progress, and retrieve results when
+they become available.
+
+Executing an {esql} query is commonly quite fast, however queries across
+large data sets or frozen data can take some time. To avoid long waits,
+run an async {esql} query.
+
+Queries initiated by the async query API may return results or not. The
+`wait_for_completion_timeout` property determines how long to wait for
+the results. If the results are not available by this time, a
+<<esql-async-query-api-response-body-query-id,query id>> is return which
+can be later used to retrieve the results. For example:
+
+[source,console]
+----
+POST /_query/async
+{
+  "query": """
+    FROM library
+    | EVAL year = DATE_TRUNC(1 YEARS, release_date)
+    | STATS MAX(page_count) BY year
+    | SORT year
+    | LIMIT 5
+  """,
+  "wait_for_completion_timeout": "2s"
+}
+----
+// TEST[setup:library]
+
+If the results are not available within the given timeout period, 2
+seconds in this case, no results are returned but rather a response that
+includes:
+
+* A query ID
+* An `is_running` value of _true_, indicating the query is ongoing
+
+The query continues to run in the background without blocking other
+requests.
+
+[source,console-result]
+----
+{
+  "id": "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
+  "is_running": true
+}
+----
+// TEST[skip: no access to query ID - may return response values]
+
+To check the progress of an async query, use the <<esql-async-query-get-api,
+{esql} async query get API>> with the query ID. Specify how long you'd like
+for complete results in the `wait_for_completion_timeout` parameter.
+
+[source,console]
+----
+GET /_query/async/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=?wait_for_completion_timeout=30s
+----
+// TEST[skip: no access to query ID - may return response values]
+
+If the response's `is_running` value is `false`, the query has finished,
+and the results are returned.
+
+[source,console-result]
+----
+{
+  "is_running": false,
+  "columns": ...
+}
+----
+// TEST[skip: no access to query ID - may return response values]
+
+Use the <<esql-async-query-delete-api, {esql} async query delete API>> to
+delete an async query before the `keep_alive` period ends. If the query
+is still running, {es} cancels it.
+
+[source,console]
+----
+DELETE /_query/async/FmdMX2pIang3UWhLRU5QS0lqdlppYncaMUpYQ05oSkpTc3kwZ21EdC1tbFJXQToxOTI=
+----
+// TEST[skip: no access to query ID]
+

docs/reference/esql/esql-using.asciidoc

@@ -2,7 +2,7 @@
 == Using {esql}
 
 <<esql-rest>>::
-Information about using the <<esql-query-api,{esql} query API>>.
+Information about using the <<esql-apis,{esql} query APIs>>.
 
 <<esql-kibana>>::
 Using {esql} in {kib} to query and aggregate your data, create visualizations,
@@ -10,7 +10,7 @@ and set up alerts.
 
 <<esql-elastic-security>>::
 Using {esql} in {elastic-sec} to investigate events in Timeline, create
-detection rules, and build {esql} queries using Elastic AI Assistant. 
+detection rules, and build {esql} queries using Elastic AI Assistant.
 
 <<esql-task-management>>::
 Using the <<tasks,task management API>> to list and cancel {esql} queries.

docs/reference/rest-api/index.asciidoc

@@ -24,7 +24,7 @@ not be included yet.
 * <<docs, Document APIs>>
 * <<enrich-apis,Enrich APIs>>
 * <<eql-apis,EQL search APIs>>
-* <<esql-query-api,{esql} query API>>
+* <<esql-apis,{esql} query APIs>>
 * <<find-structure,Find structure API>>
 * <<fleet-apis,Fleet APIs>>
 * <<graph-explore-api,Graph explore API>>
@@ -72,7 +72,7 @@ include::{es-repo-dir}/data-streams/data-stream-apis.asciidoc[]
 include::{es-repo-dir}/docs.asciidoc[]
 include::{es-repo-dir}/ingest/apis/enrich/index.asciidoc[]
 include::{es-repo-dir}/eql/eql-apis.asciidoc[]
-include::{es-repo-dir}/esql/esql-query-api.asciidoc[]
+include::{es-repo-dir}/esql/esql-apis.asciidoc[]
 include::{es-repo-dir}/features/apis/features-apis.asciidoc[]
 include::{es-repo-dir}/fleet/index.asciidoc[]
 include::{es-repo-dir}/text-structure/apis/find-structure.asciidoc[leveloffset=+1]