elasticsearch/docs/reference/upgrade/reindex_upgrade.asciidoc

[[reindex-upgrade]]
== Reindex before upgrading

Elasticsearch can read indices created in the *previous major version*.
Older indices must be reindexed or deleted. Elasticsearch 6.x
can use indices created in Elasticsearch 5.x, but not those created in
Elasticsearch 2.x or before. Elasticsearch 5.x can use indices created in
Elasticsearch 2.x, but not those created in 1.x or before.

Elasticsearch nodes will fail to start if incompatible indices are present.

To upgrade an Elasticsearch 5.x cluster that contains indices created in 2.x,
you must reindex or delete them before upgrading to 6.x.
For more information, see <<reindex-upgrade-inplace, Reindex in place>>.

To upgrade an Elasticsearch cluster running 2.x, you have two options:

* Perform a <<restart-upgrade, full cluster restart upgrade>> to 5.6,
  <<reindex-upgrade-inplace, reindex>> the 2.x indices, then perform a
  <<rolling-upgrades, rolling upgrade>> to 6.x. If your Elasticsearch 2.x
  cluster contains indices that were created before 2.x, you must either
  delete or reindex them before upgrading to 5.6. For more information about
  upgrading from 2.x to 5.6, see https://www.elastic.co/guide/en/elasticsearch/reference/5.6/setup-upgrade.html[
  Upgrading Elasticsearch] in the Elasticsearch 5.6 Reference.

* Create a new 6.x cluster and <<reindex-upgrade-remote, reindex from
  remote>> to import indices directly from the 2.x cluster.

To upgrade an Elasticsearch 1.x cluster, you have two options:

* Perform a <<restart-upgrade, full cluster restart upgrade>> to Elasticsearch
  2.4.x and <<reindex-upgrade-inplace, reindex>> or delete the 1.x indices.
  Then, perform a full cluster restart upgrade to 5.6 and reindex or delete
  the 2.x indices. Finally, perform a <<rolling-upgrades, rolling upgrade>>
  to 6.x. For more information about upgrading from 1.x to 2.4, see https://www.elastic.co/guide/en/elasticsearch/reference/2.4/setup-upgrade.html[
  Upgrading Elasticsearch] in the Elasticsearch 2.4 Reference.
  For more information about upgrading from 2.4 to 5.6, see https://www.elastic.co/guide/en/elasticsearch/reference/5.6/setup-upgrade.html[
  Upgrading Elasticsearch] in the Elasticsearch 5.6 Reference.

* Create a new 6.x cluster and <<reindex-upgrade-remote, reindex from
  remote>> to import indices directly from the 1.x cluster.

.Upgrading time-based indices
*******************************************

If you use time-based indices, you likely won't need to carry
pre-5.x indices forward to 6.x. Data in time-based indices
generally becomes less useful as time passes and are
deleted as they age past your retention period.

Unless you have an unusally long retention period, you can just
wait to upgrade to 6.x until all of your pre-5.x indices have
been deleted.

*******************************************


[[reindex-upgrade-inplace]]
=== Reindex in place

To manually reindex your old indices with the <<docs-reindex,`reindex` API>>:

. Create a new index and copy the mappings and settings from the old index.
. Set the `refresh_interval` to `-1` and the `number_of_replicas` to `0` for
  efficient reindexing.
. Reindex all documents from the old index into the new index using the
  <<docs-reindex,reindex API>>.
. Reset the `refresh_interval` and `number_of_replicas` to the values
  used in the old index.
. Wait for the index status to change to `green`.
. In a single <<indices-aliases,update aliases>> request:

.. Delete the old index.
.. Add an alias with the old index name to the new index.
.. Add any aliases that existed on the old index to the new index.


// Need to update the CSS to override sidebar titles.
[role="xpack"]
.Migration assistance and upgrade tools
*******************************************
{xpack} 5.6 provides migration assistance and upgrade tools that simplify
reindexing and upgrading to 6.x. These tools are free with the X-Pack trial
and Basic licenses and you can use them to upgrade whether or not X-Pack is a
regular part of your Elastic Stack. For more information, see
{stack-ref}/upgrading-elastic-stack.html.
*******************************************

[[reindex-upgrade-remote]]
=== Reindex from a remote cluster

You can use <<reindex-from-remote,reindex from remote>> to migrate indices from
your old cluster to a new 6.x cluster. This enables you move to 6.x from a
pre-5.6 cluster without interrupting service.

[WARNING]
=============================================

Elasticsearch provides backwards compatibility support that enables
indices from the previous major version to be upgraded to the
current major version. Skipping a major version means that you must
resolve any backward compatibility issues yourself.

=============================================

To migrate your indices:

. Set up a new 6.x cluster alongside your old cluster. Enable it to access
your old cluster by adding your old cluster to the `reindex.remote.whitelist` in `elasticsearch.yml`:
+
--
[source,yaml]
--------------------------------------------------
reindex.remote.whitelist: oldhost:9200
--------------------------------------------------

[NOTE]
=============================================
The new cluster doesn't have to start fully-scaled out. As you migrate
indices and shift the load to the new cluster, you can add nodes to the new
cluster and remove nodes from the old one.

=============================================
--

. For each index that you need to migrate to the 6.x cluster:

.. Create a new index in 6.x with the appropriate mappings and settings. Set the
  `refresh_interval` to `-1` and set `number_of_replicas` to `0` for
  faster reindexing.

.. <<reindex-from-remote,Reindex from remote>> to pull documents from the
  old index into the new 6.x index:
+
--
[source,js]
--------------------------------------------------
POST _reindex
{
  "source": {
    "remote": {
      "host": "http://oldhost:9200",
      "username": "user",
      "password": "pass"
    },
    "index": "source",
    "query": {
      "match": {
        "test": "data"
      }
    }
  },
  "dest": {
    "index": "dest"
  }
}
--------------------------------------------------
// CONSOLE
// TEST[setup:host]
// TEST[s/^/PUT source\n/]
// TEST[s/oldhost:9200",/\${host}"/]
// TEST[s/"username": "user",//]
// TEST[s/"password": "pass"//]

If you run the reindex job in the background by setting `wait_for_completion`
to `false`, the reindex request returns a `task_id` you can use to
monitor progress of the reindex job with the <<tasks,task API>>:
`GET _tasks/TASK_ID`.
--

.. When the reindex job completes, set the `refresh_interval` and
  `number_of_replicas` to the desired values (the default settings are
  `30s` and `1`).

.. Once replication is complete and the status of the new index is `green`,
  you can delete the old index.