|
@@ -7,7 +7,7 @@
|
|
|
Copies documents from a _source_ to a _destination_.
|
|
|
|
|
|
The source and destination can be any pre-existing index, index alias, or
|
|
|
-<<data-streams,data stream>>. However, the source and destination must be
|
|
|
+<<data-streams,data stream>>. However, the source and destination must be
|
|
|
different. For example, you cannot reindex a data stream into itself.
|
|
|
|
|
|
[IMPORTANT]
|
|
@@ -16,7 +16,8 @@ Reindex requires <<mapping-source-field,`_source`>> to be enabled for
|
|
|
all documents in the source.
|
|
|
|
|
|
The destination must exist and should be configured as wanted before calling `_reindex`.
|
|
|
-Reindex does not copy the settings from the source or its associated template.
|
|
|
+Reindex does not copy the settings from the source or its associated template.
|
|
|
+
|
|
|
Mappings, shard counts, replicas, and so on must be configured ahead of time.
|
|
|
=================================================
|
|
|
|
|
@@ -70,8 +71,11 @@ POST _reindex
|
|
|
[[docs-reindex-api-desc]]
|
|
|
==== {api-description-title}
|
|
|
|
|
|
-Extracts the <<mapping-source-field,document source>> from the reindex request's source and indexes the documents into the destination.
|
|
|
-You can copy all documents to the destination, or reindex a subset of the documents.
|
|
|
+// tag::docs-reindex-api-desc-tag[]
|
|
|
+Extracts the <<mapping-source-field,document source>> from the source index and indexes the documents into the destination index.
|
|
|
+You can copy all documents to the destination index, or reindex a subset of the documents.
|
|
|
+// end::docs-reindex-api-desc-tag[]
|
|
|
+
|
|
|
|
|
|
Just like <<docs-update-by-query,`_update_by_query`>>, `_reindex` gets a
|
|
|
snapshot of the source but its destination must be **different** so
|
|
@@ -95,8 +99,8 @@ any reindex request to a destination data stream must have an `op_type`
|
|
|
of`create`. A reindex can only add new documents to a destination data stream.
|
|
|
It cannot update existing documents in a destination data stream.
|
|
|
|
|
|
-By default, version conflicts abort the `_reindex` process.
|
|
|
-To continue reindexing if there are conflicts, set the `"conflicts"` request body parameter to `proceed`.
|
|
|
+By default, version conflicts abort the `_reindex` process.
|
|
|
+To continue reindexing if there are conflicts, set the `"conflicts"` request body parameter to `proceed`.
|
|
|
In this case, the response includes a count of the version conflicts that were encountered.
|
|
|
Note that the handling of other error types is unaffected by the `"conflicts"` parameter.
|
|
|
|
|
@@ -105,9 +109,9 @@ Note that the handling of other error types is unaffected by the `"conflicts"` p
|
|
|
|
|
|
If the request contains `wait_for_completion=false`, {es}
|
|
|
performs some preflight checks, launches the request, and returns a
|
|
|
-<<tasks,`task`>> you can use to cancel or get the status of the task.
|
|
|
-{es} creates a record of this task as a document at `.tasks/task/${taskId}`.
|
|
|
-When you are done with a task, you should delete the task document so
|
|
|
+<<tasks,`task`>> you can use to cancel or get the status of the task.
|
|
|
+{es} creates a record of this task as a document at `.tasks/task/${taskId}`.
|
|
|
+When you are done with a task, you should delete the task document so
|
|
|
{es} can reclaim the space.
|
|
|
|
|
|
[[docs-reindex-from-multiple-sources]]
|
|
@@ -141,7 +145,7 @@ done
|
|
|
|
|
|
Set `requests_per_second` to any positive decimal number (`1.4`, `6`,
|
|
|
`1000`, etc.) to throttle the rate at which `_reindex` issues batches of index
|
|
|
-operations. Requests are throttled by padding each batch with a wait time.
|
|
|
+operations. Requests are throttled by padding each batch with a wait time.
|
|
|
To disable throttling, set `requests_per_second` to `-1`.
|
|
|
|
|
|
The throttling is done by waiting between batches so that the `scroll` that `_reindex`
|
|
@@ -156,7 +160,7 @@ target_time = 1000 / 500 per second = 2 seconds
|
|
|
wait_time = target_time - write_time = 2 seconds - .5 seconds = 1.5 seconds
|
|
|
--------------------------------------------------
|
|
|
|
|
|
-Since the batch is issued as a single `_bulk` request, large batch sizes
|
|
|
+Since the batch is issued as a single `_bulk` request, large batch sizes
|
|
|
cause Elasticsearch to create many requests and then wait for a while before
|
|
|
starting the next set. This is "bursty" instead of "smooth".
|
|
|
|
|
@@ -447,13 +451,13 @@ include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=max_docs]
|
|
|
==== {api-request-body-title}
|
|
|
|
|
|
`conflicts`::
|
|
|
-(Optional, enum) Set to `proceed` to continue reindexing even if there are conflicts.
|
|
|
+(Optional, enum) Set to `proceed` to continue reindexing even if there are conflicts.
|
|
|
Defaults to `abort`.
|
|
|
|
|
|
`source`::
|
|
|
`index`:::
|
|
|
-(Required, string) The name of the data stream, index, or index alias you are copying _from_.
|
|
|
-Also accepts a comma-separated list to reindex from multiple sources.
|
|
|
+(Required, string) The name of the data stream, index, or index alias you are copying _from_.
|
|
|
+Also accepts a comma-separated list to reindex from multiple sources.
|
|
|
|
|
|
`max_docs`:::
|
|
|
(Optional, integer) The maximum number of documents to reindex.
|
|
@@ -468,22 +472,22 @@ Required when indexing from remote.
|
|
|
`username`::::
|
|
|
(Optional, string) The username to use for authentication with the remote host.
|
|
|
`password`::::
|
|
|
-(Optional, string) The password to use for authentication with the remote host.
|
|
|
-`socket_timeout`::::
|
|
|
+(Optional, string) The password to use for authentication with the remote host.
|
|
|
+`socket_timeout`::::
|
|
|
(Optional, <<time-units, time units>>) The remote socket read timeout. Defaults to 30 seconds.
|
|
|
-`connect_timeout`::::
|
|
|
+`connect_timeout`::::
|
|
|
(Optional, <<time-units, time units>>) The remote connection timeout. Defaults to 30 seconds.
|
|
|
|
|
|
`size`:::
|
|
|
-{Optional, integer) The number of documents to index per batch.
|
|
|
-Use when indexing from remote to ensure that the batches fit within the on-heap buffer,
|
|
|
-which defaults to a maximum size of 100 MB.
|
|
|
+{Optional, integer) The number of documents to index per batch.
|
|
|
+Use when indexing from remote to ensure that the batches fit within the on-heap buffer,
|
|
|
+which defaults to a maximum size of 100 MB.
|
|
|
|
|
|
`slice`:::
|
|
|
`id`::::
|
|
|
-(Optional, integer) Slice ID for <<docs-reindex-manual-slice, manual slicing>>.
|
|
|
+(Optional, integer) Slice ID for <<docs-reindex-manual-slice, manual slicing>>.
|
|
|
`max`::::
|
|
|
-(Optional, integer) Total number of slices.
|
|
|
+(Optional, integer) Total number of slices.
|
|
|
|
|
|
`sort`:::
|
|
|
+
|
|
@@ -495,31 +499,31 @@ deprecated::[7.6, Sort in reindex is deprecated. Sorting in reindex was never gu
|
|
|
--
|
|
|
|
|
|
`_source`:::
|
|
|
-(Optional, string) If `true` reindexes all source fields.
|
|
|
-Set to a list to reindex select fields.
|
|
|
-Defaults to `true`.
|
|
|
+(Optional, string) If `true` reindexes all source fields.
|
|
|
+Set to a list to reindex select fields.
|
|
|
+Defaults to `true`.
|
|
|
|
|
|
`dest`::
|
|
|
`index`:::
|
|
|
(Required, string) The name of the data stream, index, or index alias you are copying _to_.
|
|
|
|
|
|
`version_type`:::
|
|
|
-(Optional, enum) The versioning to use for the indexing operation.
|
|
|
-Valid values: `internal`, `external`, `external_gt`, `external_gte`.
|
|
|
+(Optional, enum) The versioning to use for the indexing operation.
|
|
|
+Valid values: `internal`, `external`, `external_gt`, `external_gte`.
|
|
|
See <<index-version-types>> for more information.
|
|
|
|
|
|
-`op_type`:::
|
|
|
-(Optional, enum) Set to create to only index documents that do not already exist (put if absent).
|
|
|
+`op_type`:::
|
|
|
+(Optional, enum) Set to create to only index documents that do not already exist (put if absent).
|
|
|
Valid values: `index`, `create`. Defaults to `index`.
|
|
|
+
|
|
|
IMPORTANT: To reindex to a data stream destination, this argument must be
|
|
|
`create`.
|
|
|
|
|
|
`script`::
|
|
|
-`source`:::
|
|
|
-(Optional, string) The script to run to update the document source or metadata when reindexing.
|
|
|
+`source`:::
|
|
|
+(Optional, string) The script to run to update the document source or metadata when reindexing.
|
|
|
`lang`:::
|
|
|
-(Optional, enum) The script language: `painless`, `expression`, `mustache`, `java`.
|
|
|
+(Optional, enum) The script language: `painless`, `expression`, `mustache`, `java`.
|
|
|
For more information, see <<modules-scripting>>.
|
|
|
|
|
|
|
|
@@ -623,7 +627,7 @@ POST _reindex
|
|
|
[[docs-reindex-select-max-docs]]
|
|
|
===== Reindex select documents with `max_docs`
|
|
|
|
|
|
-You can limit the number of processed documents by setting `max_docs`.
|
|
|
+You can limit the number of processed documents by setting `max_docs`.
|
|
|
For example, this request copies a single document from `twitter` to
|
|
|
`new_twitter`:
|
|
|
|
|
@@ -645,7 +649,7 @@ POST _reindex
|
|
|
[[docs-reindex-multiple-sources]]
|
|
|
===== Reindex from multiple sources
|
|
|
|
|
|
-The `index` attribute in `source` can be a list, allowing you to copy from lots
|
|
|
+The `index` attribute in `source` can be a list, allowing you to copy from lots
|
|
|
of sources in one request. This will copy documents from the
|
|
|
`twitter` and `blog` indices:
|
|
|
|
|
@@ -1031,7 +1035,7 @@ It is not possible to configure SSL in the body of the `_reindex` request.
|
|
|
The following settings are supported:
|
|
|
|
|
|
`reindex.ssl.certificate_authorities`::
|
|
|
-List of paths to PEM encoded certificate files that should be trusted.
|
|
|
+List of paths to PEM encoded certificate files that should be trusted.
|
|
|
You cannot specify both `reindex.ssl.certificate_authorities` and
|
|
|
`reindex.ssl.truststore.path`.
|
|
|
|
|
@@ -1056,7 +1060,7 @@ or "pkcs12", this setting defaults to `PKCS12`. Otherwise, it defaults to `jks`.
|
|
|
|
|
|
`reindex.ssl.verification_mode`::
|
|
|
Indicates the type of verification to protect against man in the middle attacks
|
|
|
-and certificate forgery.
|
|
|
+and certificate forgery.
|
|
|
One of `full` (verify the hostname and the certificate path), `certificate`
|
|
|
(verify the certificate path, but not the hostname) or `none` (perform no
|
|
|
verification - this is strongly discouraged in production environments).
|
|
@@ -1076,12 +1080,12 @@ You cannot specify both `reindex.ssl.key` and `reindex.ssl.keystore.path`.
|
|
|
`reindex.ssl.key_passphrase`::
|
|
|
Specifies the passphrase to decrypt the PEM encoded private key
|
|
|
(`reindex.ssl.key`) if it is encrypted.
|
|
|
-Cannot be used with `reindex.ssl.secure_key_passphrase`.
|
|
|
+Cannot be used with `reindex.ssl.secure_key_passphrase`.
|
|
|
|
|
|
`reindex.ssl.secure_key_passphrase` (<<secure-settings,Secure>>)::
|
|
|
Specifies the passphrase to decrypt the PEM encoded private key
|
|
|
(`reindex.ssl.key`) if it is encrypted.
|
|
|
-Cannot be used with `reindex.ssl.key_passphrase`.
|
|
|
+Cannot be used with `reindex.ssl.key_passphrase`.
|
|
|
|
|
|
`reindex.ssl.keystore.path`::
|
|
|
Specifies the path to the keystore that contains a private key and certificate
|
|
@@ -1091,11 +1095,11 @@ You cannot specify both `reindex.ssl.key` and `reindex.ssl.keystore.path`.
|
|
|
|
|
|
`reindex.ssl.keystore.type`::
|
|
|
The type of the keystore (`reindex.ssl.keystore.path`). Must be either `jks` or `PKCS12`.
|
|
|
-If the keystore path ends in ".p12", ".pfx" or "pkcs12", this setting defaults
|
|
|
+If the keystore path ends in ".p12", ".pfx" or "pkcs12", this setting defaults
|
|
|
to `PKCS12`. Otherwise, it defaults to `jks`.
|
|
|
|
|
|
`reindex.ssl.keystore.password`::
|
|
|
-The password to the keystore (`reindex.ssl.keystore.path`). This setting cannot be used
|
|
|
+The password to the keystore (`reindex.ssl.keystore.path`). This setting cannot be used
|
|
|
with `reindex.ssl.keystore.secure_password`.
|
|
|
|
|
|
`reindex.ssl.keystore.secure_password` (<<secure-settings,Secure>>)::
|
|
@@ -1104,10 +1108,10 @@ This setting cannot be used with `reindex.ssl.keystore.password`.
|
|
|
|
|
|
`reindex.ssl.keystore.key_password`::
|
|
|
The password for the key in the keystore (`reindex.ssl.keystore.path`).
|
|
|
-Defaults to the keystore password. This setting cannot be used with
|
|
|
+Defaults to the keystore password. This setting cannot be used with
|
|
|
`reindex.ssl.keystore.secure_key_password`.
|
|
|
|
|
|
`reindex.ssl.keystore.secure_key_password` (<<secure-settings,Secure>>)::
|
|
|
The password for the key in the keystore (`reindex.ssl.keystore.path`).
|
|
|
-Defaults to the keystore password. This setting cannot be used with
|
|
|
+Defaults to the keystore password. This setting cannot be used with
|
|
|
`reindex.ssl.keystore.key_password`.
|