elasticsearch/docs/reference/ccr/requirements.asciidoc

[role="xpack"]
[testenv="platinum"]
[[ccr-requirements]]
=== Requirements for leader indices

beta[]

Cross-cluster replication works by replaying the history of individual write
operations that were performed on the shards of the leader index. This means that the
history of these operations needs to be retained on the leader shards so that
they can be pulled by the follower shard tasks. The underlying mechanism used to
retain these operations is _soft deletes_. A soft delete occurs whenever an
existing document is deleted or updated. By retaining these soft deletes up to
configurable limits, the history of operations can be retained on the leader
shards and made available to the follower shard tasks as it replays the history
of operations.

Soft deletes must be enabled for indices that you want to use as leader
indices. Enabling soft deletes requires the addition of some index settings at
index creation time. You must add these settings to your create index
requests or to the index templates that you use to manage the creation of new
indices.

IMPORTANT: This means that {ccr} can not be used on existing indices. If you have
existing data that you want to replicate from another cluster, you must
{ref}/docs-reindex.html[reindex] your data into a new index with soft deletes 
enabled.

[float]
[[ccr-overview-soft-deletes]]
==== Soft delete settings

`index.soft_deletes.enabled`::

Whether or not soft deletes are enabled on the index. Soft deletes can only be
configured at index creation and only on indices created on or after 6.5.0. The
default value is `false`.

`index.soft_deletes.retention.operations`::

The number of soft deletes to retain. Soft deletes are collected during merges
on the underlying Lucene index yet retained up to the number of operations
configured by this setting. The default value is `0`.

For more information about index settings, see {ref}/index-modules.html[Index modules].

[float]
[[ccr-overview-beats]]
==== Setting soft deletes on indices created by APM Server or Beats

If you want to replicate indices created by APM Server or Beats, and are
allowing APM Server or Beats to manage index templates, you need to enable
soft deletes on the underlying index templates. To enable soft deletes on the
underlying index templates, add the following changes to the relevant APM Server
or Beats configuration file.

["source","yaml"]
----------------------------------------------------------------------
setup.template.overwrite: true
setup.template.settings:
  index.soft_deletes.enabled: true
  index.soft_deletes.retention.operations: 1024
----------------------------------------------------------------------

For additional information on controlling the index templates managed by APM
Server or Beats, see the relevant documentation on loading the Elasticsearch
index template.