|
|
@@ -1,843 +0,0 @@
|
|
|
-[role="xpack"]
|
|
|
-[testenv="basic"]
|
|
|
-[[ilm-policy-definition]]
|
|
|
-== Policy phases and actions
|
|
|
-
|
|
|
-There are four stages in the index lifecycle, in the order
|
|
|
-they are executed.
|
|
|
-
|
|
|
-[options="header"]
|
|
|
-|======
|
|
|
-| Name | Description
|
|
|
-| `hot` | The index is actively being written to
|
|
|
-| `warm` | The index is generally not being written to, but is still queried
|
|
|
-| `cold` | The index is no longer being updated and is seldom queried. The
|
|
|
-information still needs to be searchable, but it's okay if those queries are
|
|
|
-slower.
|
|
|
-| `delete` | The index is no longer needed and can safely be deleted
|
|
|
-|======
|
|
|
-
|
|
|
-Each of these stages is called a "phase". A policy does not need to configure
|
|
|
-each phase for an index. For example, one policy may define only the hot
|
|
|
-phase and the delete phase, while another may define all four phases.
|
|
|
-
|
|
|
-=== Timing
|
|
|
-
|
|
|
-Indices enter phases based on a phase's `min_age` parameter.
|
|
|
-The index will not enter the phase until the index's age is older than that
|
|
|
-of the `min_age`. The parameter is configured using a time
|
|
|
-duration format (see <<time-units, Time Units>>).
|
|
|
-
|
|
|
-`min_age` defaults to zero seconds `0s` for each phase if not specified.
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT _ilm/policy/my_policy
|
|
|
-{
|
|
|
- "policy": {
|
|
|
- "phases": {
|
|
|
- "warm": {
|
|
|
- "min_age": "1d",
|
|
|
- "actions": {
|
|
|
- "allocate": {
|
|
|
- "number_of_replicas": 1
|
|
|
- }
|
|
|
- }
|
|
|
- },
|
|
|
- "delete": {
|
|
|
- "min_age": "30d",
|
|
|
- "actions": {
|
|
|
- "delete": {}
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-
|
|
|
-The above example configures a policy that moves the index into the warm
|
|
|
-phase after one day. Until then, the index is in a waiting state. After
|
|
|
-moving into the warm phase, it will wait until 30 days have elapsed before
|
|
|
-moving to the delete phase and deleting the index.
|
|
|
-
|
|
|
-`min_age` is usually the time elapsed from the time the index is created, unless
|
|
|
-the `index.lifecycle.origination_date` index setting is configured, in which
|
|
|
-case the `min_age` will be the time elapsed since that specified date. If the
|
|
|
-index is rolled over, then `min_age` is the time elapsed from the time the
|
|
|
-index is rolled over. The intention here is to execute following phases and
|
|
|
-actions relative to when data was written last to a rolled over index.
|
|
|
-
|
|
|
-The previous phase's actions must complete before {ilm} will check `min_age` and
|
|
|
-transition into the next phase. By default, {ilm} checks for indices that meet
|
|
|
-policy criteria, like `min_age`, every 10 minutes. You can use the
|
|
|
-`indices.lifecycle.poll_interval` cluster setting to control how often this
|
|
|
-check occurs.
|
|
|
-
|
|
|
-=== Phase Execution
|
|
|
-
|
|
|
-The current phase definition, of an index's policy being executed, is stored
|
|
|
-in the index's metadata. This phase definition is cached to prevent changes to
|
|
|
-the policy from putting the index in a state where it cannot proceed from its
|
|
|
-current step. When the policy is updated we check to see if this phase
|
|
|
-definition can be safely updated, and if so, update the cached definition in
|
|
|
-indices using the updated policy. The phase and its actions are compiled into a
|
|
|
-series of discrete steps that are executed sequentially. Since some {ilm-init}
|
|
|
-actions are more complex and involve multiple operations against an index, each
|
|
|
-of these operations are done in isolation in a unit called a "step". The
|
|
|
-<<ilm-explain-lifecycle,Explain Lifecycle API>> exposes this information to us
|
|
|
-to see which step our index is either to execute next, or is currently
|
|
|
-executing.
|
|
|
-
|
|
|
-=== Actions
|
|
|
-
|
|
|
-The below list shows the actions which are available in each phase.
|
|
|
-
|
|
|
-NOTE: The order that configured actions are performed in within each phase is
|
|
|
-determined automatically by {ilm-init}, and cannot be changed by changing the
|
|
|
-policy definition.
|
|
|
-
|
|
|
-* Hot
|
|
|
- - <<ilm-set-priority-action,Set Priority>>
|
|
|
- - <<ilm-unfollow-action,Unfollow>>
|
|
|
- - <<ilm-rollover-action,Rollover>>
|
|
|
-* Warm
|
|
|
- - <<ilm-set-priority-action,Set Priority>>
|
|
|
- - <<ilm-unfollow-action,Unfollow>>
|
|
|
- - <<ilm-readonly-action,Read-Only>>
|
|
|
- - <<ilm-allocate-action,Allocate>>
|
|
|
- - <<ilm-shrink-action,Shrink>>
|
|
|
- - <<ilm-forcemerge-action,Force Merge>>
|
|
|
-* Cold
|
|
|
- - <<ilm-set-priority-action,Set Priority>>
|
|
|
- - <<ilm-unfollow-action,Unfollow>>
|
|
|
- - <<ilm-allocate-action,Allocate>>
|
|
|
- - <<ilm-freeze-action,Freeze>>
|
|
|
- - <<ilm-searchable-snapshot-action, Searchable Snapshot>>
|
|
|
-* Delete
|
|
|
- - <<ilm-wait-for-snapshot-action,Wait For Snapshot>>
|
|
|
- - <<ilm-delete-action,Delete>>
|
|
|
-
|
|
|
-[[ilm-allocate-action]]
|
|
|
-==== Allocate
|
|
|
-
|
|
|
-Phases allowed: warm, cold.
|
|
|
-
|
|
|
-The Allocate action allows you to specify which nodes are allowed to host the
|
|
|
-shards of the index and set the number of replicas.
|
|
|
-Behind the scenes, it is modifying the index settings
|
|
|
-for shard filtering and/or replica counts. When updating the number of replicas,
|
|
|
-configuring allocation rules is optional. When configuring allocation rules,
|
|
|
-setting number of replicas is optional. Although this action can be treated as
|
|
|
-two separate index settings updates, both can be configured at once.
|
|
|
-
|
|
|
-For more information about how {es} uses replicas for scaling, see
|
|
|
-<<scalability>>. See <<shard-allocation-filtering>> for more information about
|
|
|
-controlling where Elasticsearch allocates shards of a particular index.
|
|
|
-
|
|
|
---
|
|
|
-NOTE: As allocate action is not allowed in `hot` phase, the initial allocation for the index should be done manually or via index templates, as ILM won't take care of index allocation during `hot` phase.
|
|
|
-
|
|
|
---
|
|
|
-
|
|
|
-[[ilm-allocate-options]]
|
|
|
-.Allocate Options
|
|
|
-[options="header"]
|
|
|
-|======
|
|
|
-| Name | Required | Default | Description
|
|
|
-| `number_of_replicas` | no | - | The number of replicas to
|
|
|
- assign to the index
|
|
|
-| `include` | no | - | assigns an index to nodes
|
|
|
- having at least _one_ of the attributes
|
|
|
-| `exclude` | no | - | assigns an index to nodes having
|
|
|
- _none_ of the attributes
|
|
|
-| `require` | no | - | assigns an index to nodes having
|
|
|
- _all_ of the attributes
|
|
|
-|======
|
|
|
-
|
|
|
-If `number_of_replicas` is not configured, then at least one of `include`,
|
|
|
-`exclude`, and `require` is required. An empty Allocate Action with no configuration
|
|
|
-is invalid.
|
|
|
-
|
|
|
-===== Example: Change number of replicas
|
|
|
-
|
|
|
-In this example, the index's number of replicas is changed to `2`, while allocation
|
|
|
-rules are unchanged.
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT _ilm/policy/my_policy
|
|
|
-{
|
|
|
- "policy": {
|
|
|
- "phases": {
|
|
|
- "warm": {
|
|
|
- "actions": {
|
|
|
- "allocate" : {
|
|
|
- "number_of_replicas" : 2
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-
|
|
|
-===== Example: Assign index to node with specific "box_type" attribute
|
|
|
-
|
|
|
-This example assigns the index to nodes with `box_type` attribute of "hot" or "warm".
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT _ilm/policy/my_policy
|
|
|
-{
|
|
|
- "policy": {
|
|
|
- "phases": {
|
|
|
- "warm": {
|
|
|
- "actions": {
|
|
|
- "allocate" : {
|
|
|
- "include" : {
|
|
|
- "box_type": "hot,warm"
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-
|
|
|
-This depends on `hot` and `warm` attributes specified in the node configurations. For example, to designate a node as a _hot_ node, set `node.attr.data: hot` in `elasticsearch.yml`. For more information, see <<index-allocation-filters>>.
|
|
|
-
|
|
|
-===== Example: Assign index to a specific node and update replica settings
|
|
|
-
|
|
|
-This example updates the index to have one replica per shard and be allocated
|
|
|
-to nodes with a `box_type` attribute of "cold".
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT _ilm/policy/my_policy
|
|
|
-{
|
|
|
- "policy": {
|
|
|
- "phases": {
|
|
|
- "warm": {
|
|
|
- "actions": {
|
|
|
- "allocate" : {
|
|
|
- "number_of_replicas": 1,
|
|
|
- "require" : {
|
|
|
- "box_type": "cold"
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-
|
|
|
-[[ilm-wait-for-snapshot-action]]
|
|
|
-==== Wait For Snapshot
|
|
|
-
|
|
|
-Phases allowed: delete.
|
|
|
-
|
|
|
-The Wait For Snapshot Action waits for defined SLM policy to be executed to ensure that snapshot of index exists before
|
|
|
-deletion.
|
|
|
-
|
|
|
-[[ilm-wait-for-snapshot-options]]
|
|
|
-.Wait For Snapshot
|
|
|
-[options="header"]
|
|
|
-|======
|
|
|
-| Name | Required | Default | Description
|
|
|
-| `policy` | yes | - | SLM policy name that this action should wait for
|
|
|
-|======
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT _ilm/policy/my_policy
|
|
|
-{
|
|
|
- "policy": {
|
|
|
- "phases": {
|
|
|
- "delete": {
|
|
|
- "actions": {
|
|
|
- "wait_for_snapshot" : {
|
|
|
- "policy": "slm-policy-name"
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-
|
|
|
-[[ilm-delete-action]]
|
|
|
-==== Delete
|
|
|
-
|
|
|
-Phases allowed: delete.
|
|
|
-
|
|
|
-The Delete Action does just that, it deletes the index.
|
|
|
-
|
|
|
-[[ilm-delete-action-options]]
|
|
|
-.Delete
|
|
|
-[options="header"]
|
|
|
-|======
|
|
|
-| Name | Required | Default | Description
|
|
|
-| `delete_searchable_snapshot` | no | true | Deletes the searchable snapshot created in the cold phase, if a
|
|
|
- snapshot was created (ie. if the <<ilm-searchable-snapshot-action,
|
|
|
- Searchable Snapshot Action>> was used in the cold phase)
|
|
|
-|======
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT _ilm/policy/my_policy
|
|
|
-{
|
|
|
- "policy": {
|
|
|
- "phases": {
|
|
|
- "delete": {
|
|
|
- "actions": {
|
|
|
- "delete" : { }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-
|
|
|
-[[ilm-forcemerge-action]]
|
|
|
-==== Force Merge
|
|
|
-
|
|
|
-Phases allowed: hot, warm.
|
|
|
-
|
|
|
-NOTE: Index will be be made read-only when this action is run
|
|
|
-(see: <<dynamic-index-settings,index.blocks.write>>)
|
|
|
-
|
|
|
-NOTE: The `forcemerge` action is best effort. It might happen that some of the
|
|
|
-shards are relocating, in which case they will not be merged.
|
|
|
-
|
|
|
-NOTE: If the `forcemerge` action is used in the `hot` phase, the `rollover` action *must* be present.
|
|
|
-ILM validates this predicate and will refuse a policy with a forcemerge in the hot phase without a
|
|
|
-rollover action.
|
|
|
-
|
|
|
-The Force Merge Action <<indices-forcemerge,force merges>> the index into at
|
|
|
-most a specific number of <<indices-segments,segments>>.
|
|
|
-
|
|
|
-[[ilm-forcemerge-options]]
|
|
|
-.Force Merge Options
|
|
|
-[options="header"]
|
|
|
-|======
|
|
|
-| Name | Required | Default | Description
|
|
|
-| `max_num_segments` | yes | - | The number of segments to merge to. To fully merge the index, set it to `1`
|
|
|
-| `codec` | no | - | Optional specification of the `best_compression` codec
|
|
|
-|======
|
|
|
-
|
|
|
-[WARNING]
|
|
|
-======
|
|
|
-When using the `"codec": "best_compression"` configuration in the ILM forcemerge action, ILM will
|
|
|
-<<indices-close,close>> and then <<indices-open-close,re-open>> the index prior to a forcemerge.
|
|
|
-During this time the index will be unavailable for either read or write operations.
|
|
|
-======
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT _ilm/policy/my_policy
|
|
|
-{
|
|
|
- "policy": {
|
|
|
- "phases": {
|
|
|
- "warm": {
|
|
|
- "actions": {
|
|
|
- "forcemerge" : {
|
|
|
- "max_num_segments": 1
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-
|
|
|
-[[ilm-freeze-action]]
|
|
|
-==== Freeze
|
|
|
-
|
|
|
-Phases allowed: cold.
|
|
|
-
|
|
|
-This action will <<frozen-indices, freeze>> the index
|
|
|
-by calling the <<freeze-index-api, Freeze Index API>>.
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT _ilm/policy/my_policy
|
|
|
-{
|
|
|
- "policy": {
|
|
|
- "phases": {
|
|
|
- "cold": {
|
|
|
- "actions": {
|
|
|
- "freeze" : { }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-
|
|
|
-[IMPORTANT]
|
|
|
-================================
|
|
|
- Freezing an index will close the index and reopen it within the same API call.
|
|
|
- This causes primaries to not be allocated for a short amount of time and
|
|
|
- causes the cluster to go red until the primaries are allocated again.
|
|
|
- This limitation might be removed in the future.
|
|
|
-================================
|
|
|
-
|
|
|
-[[ilm-readonly-action]]
|
|
|
-==== Read-Only
|
|
|
-
|
|
|
-Phases allowed: warm.
|
|
|
-
|
|
|
-This action will set the index to be read-only
|
|
|
-(see: <<dynamic-index-settings,index.blocks.write>>)
|
|
|
-
|
|
|
-This action does not have any options associated with it.
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT _ilm/policy/my_policy
|
|
|
-{
|
|
|
- "policy": {
|
|
|
- "phases": {
|
|
|
- "warm": {
|
|
|
- "actions": {
|
|
|
- "readonly" : { }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-
|
|
|
-[[ilm-rollover-action]]
|
|
|
-==== Rollover
|
|
|
-
|
|
|
-Phases allowed: hot.
|
|
|
-
|
|
|
-[WARNING]
|
|
|
-index format must match pattern '^.*-\\d+$', for example (`logs-000001`).
|
|
|
-[WARNING]
|
|
|
-The managed index must set `index.lifecycle.rollover_alias` as the
|
|
|
-alias to rollover. The index must also be the write index for the alias.
|
|
|
-
|
|
|
-[IMPORTANT]
|
|
|
-If a policy using the Rollover action is used on a <<ccr-put-follow,follower
|
|
|
-index>>, policy execution will wait until the leader index rolls over (or has
|
|
|
-<<skipping-rollover, otherwise been marked as complete>>), then convert the
|
|
|
-follower index into a regular index as if <<ilm-unfollow-action,the Unfollow
|
|
|
-action>> had been used instead of rolling over.
|
|
|
-
|
|
|
-For example, if an index to be managed has an alias `my_data`. The managed
|
|
|
-index "my_index-000001" must be the write index for the alias. For more information, read
|
|
|
-<<indices-rollover-is-write-index,Write Index Alias Behavior>>.
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT my_index-000001
|
|
|
-{
|
|
|
- "settings": {
|
|
|
- "index.lifecycle.name": "my_policy",
|
|
|
- "index.lifecycle.rollover_alias": "my_data"
|
|
|
- },
|
|
|
- "aliases": {
|
|
|
- "my_data": {
|
|
|
- "is_write_index": true
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-
|
|
|
-The Rollover Action rolls an alias over to a new index when the
|
|
|
-existing index meets one of the rollover conditions.
|
|
|
-
|
|
|
-
|
|
|
-[[ilm-rollover-options]]
|
|
|
-.Rollover Options
|
|
|
-[options="header"]
|
|
|
-|======
|
|
|
-| Name | Required | Default | Description
|
|
|
-| `max_size` | no | - | max primary shard index storage size.
|
|
|
- See <<byte-units, Byte Units>>
|
|
|
- for formatting
|
|
|
-| `max_docs` | no | - | max number of documents an
|
|
|
- index is to contain before
|
|
|
- rolling over.
|
|
|
-| `max_age` | no | - | max time elapsed from index
|
|
|
- creation. See
|
|
|
- <<time-units, Time Units>>
|
|
|
- for formatting
|
|
|
-|======
|
|
|
-
|
|
|
-At least one of `max_size`, `max_docs`, `max_age` or any combinations of the
|
|
|
-three are required to be specified.
|
|
|
-
|
|
|
-===== Example: Rollover when index is too large
|
|
|
-
|
|
|
-This example rolls the index over when it is at least 100 gigabytes.
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT _ilm/policy/my_policy
|
|
|
-{
|
|
|
- "policy": {
|
|
|
- "phases": {
|
|
|
- "hot": {
|
|
|
- "actions": {
|
|
|
- "rollover" : {
|
|
|
- "max_size": "100GB"
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-
|
|
|
-===== Example: Rollover when index has too many documents
|
|
|
-
|
|
|
-This example rolls the index over when it contains at least
|
|
|
-100000000 documents.
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT _ilm/policy/my_policy
|
|
|
-{
|
|
|
- "policy": {
|
|
|
- "phases": {
|
|
|
- "hot": {
|
|
|
- "actions": {
|
|
|
- "rollover" : {
|
|
|
- "max_docs": 100000000
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-
|
|
|
-===== Example: Rollover when index is too old
|
|
|
-
|
|
|
-This example rolls the index over when it has been created at least
|
|
|
-7 days ago.
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT _ilm/policy/my_policy
|
|
|
-{
|
|
|
- "policy": {
|
|
|
- "phases": {
|
|
|
- "hot": {
|
|
|
- "actions": {
|
|
|
- "rollover" : {
|
|
|
- "max_age": "7d"
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-
|
|
|
-===== Example: Rollover when index is too old or too large
|
|
|
-
|
|
|
-This example rolls the index over when it has been created at least
|
|
|
-7 days ago or it is at least 100 gigabytes. In this case, the index will be
|
|
|
-rolled over when any of the conditions is met.
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT _ilm/policy/my_policy
|
|
|
-{
|
|
|
- "policy": {
|
|
|
- "phases": {
|
|
|
- "hot": {
|
|
|
- "actions": {
|
|
|
- "rollover" : {
|
|
|
- "max_age": "7d",
|
|
|
- "max_size": "100GB"
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-
|
|
|
-===== Example: Rollover condition stalls phase transition
|
|
|
-
|
|
|
-The Rollover action will only complete once one of its conditions is
|
|
|
-met. This means that any proceeding phases will be blocked until Rollover
|
|
|
-succeeds.
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT /_ilm/policy/rollover_policy
|
|
|
-{
|
|
|
- "policy": {
|
|
|
- "phases": {
|
|
|
- "hot": {
|
|
|
- "actions": {
|
|
|
- "rollover": {
|
|
|
- "max_size": "50G"
|
|
|
- }
|
|
|
- }
|
|
|
- },
|
|
|
- "delete": {
|
|
|
- "min_age": "1d",
|
|
|
- "actions": {
|
|
|
- "delete": {}
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-
|
|
|
-The above example illustrates a policy which attempts to delete an
|
|
|
-index one day after the index has been rolled over. It does not
|
|
|
-delete the index one day after it has been created.
|
|
|
-
|
|
|
-[[ilm-searchable-snapshot-action]]
|
|
|
-==== Searchable Snapshot
|
|
|
-
|
|
|
-Phases allowed: cold.
|
|
|
-
|
|
|
-This action takes a snapshot of the managed index in the configured repository
|
|
|
-and mounts it as a searchable snapshot.
|
|
|
-
|
|
|
-[[ilm-searchable-snapshot-options]]
|
|
|
-.Searchable Snapshot Options
|
|
|
-[options="header"]
|
|
|
-|======
|
|
|
-| Name | Required | Default | Description
|
|
|
-| `snapshot_repository` | yes | - | Repository used to store the snapshot created by this action. The snapshot will be,
|
|
|
- by default, deleted by the <<ilm-delete-action, Delete Action>> in the delete phase, if
|
|
|
- configured, but the user can configure the <<ilm-delete-action, Delete Action>> to keep
|
|
|
- the snapshot using the `delete_searchable_snapshot` option.
|
|
|
-
|
|
|
-|======
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT _ilm/policy/my_policy
|
|
|
-{
|
|
|
- "policy": {
|
|
|
- "phases": {
|
|
|
- "cold": {
|
|
|
- "actions": {
|
|
|
- "searchable_snapshot" : {
|
|
|
- "snapshot_repository" : "backing_repo"
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-[[ilm-set-priority-action]]
|
|
|
-==== Set Priority
|
|
|
-
|
|
|
-Phases allowed: hot, warm, cold.
|
|
|
-
|
|
|
-This action sets the <<recovery-prioritization, index priority>> on the index as
|
|
|
-soon as the policy enters the hot, warm, or cold phase. Indices with a higher
|
|
|
-priority will be recovered before indices with lower priorities following a node
|
|
|
-restart. Generally, indexes in the hot phase should have the highest value and
|
|
|
-indexes in the cold phase should have the lowest values. For example:
|
|
|
-100 for the hot phase, 50 for the warm phase, and 0 for the cold phase.
|
|
|
-Indicies that don't set this value have an implicit default priority of 1.
|
|
|
-
|
|
|
-[[ilm-set-priority-options]]
|
|
|
-.Set Priority Options
|
|
|
-[options="header"]
|
|
|
-|======
|
|
|
-| Name | Required | Default | Description
|
|
|
-| `priority` | yes | - | The priority for the index. Must be 0 or greater.
|
|
|
- The value may also be set to null to remove the priority.
|
|
|
-
|
|
|
-|======
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT _ilm/policy/my_policy
|
|
|
-{
|
|
|
- "policy": {
|
|
|
- "phases": {
|
|
|
- "warm": {
|
|
|
- "actions": {
|
|
|
- "set_priority" : {
|
|
|
- "priority": 50
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-
|
|
|
-[[ilm-shrink-action]]
|
|
|
-==== Shrink
|
|
|
-
|
|
|
-NOTE: Index will be be made read-only when this action is run
|
|
|
-(see: <<dynamic-index-settings,index.blocks.write>>)
|
|
|
-[IMPORTANT]
|
|
|
-If a policy using the Shrink action is used on a <<ccr-put-follow,follower
|
|
|
-index>>, policy execution will wait until the leader index rolls over (or has
|
|
|
-<<skipping-rollover, otherwise been marked as complete>>), then convert the
|
|
|
-follower index into a regular index as if <<ilm-unfollow-action,the Unfollow
|
|
|
-action>> had been used before shrink is applied, as shrink cannot be safely
|
|
|
-applied to follower indices.
|
|
|
-
|
|
|
-This action shrinks an existing index into a new index with fewer primary
|
|
|
-shards. It calls the <<indices-shrink-index,Shrink API>> to shrink the index.
|
|
|
-Since allocating all the primary shards of the index to one node is a
|
|
|
-prerequisite, this action will first allocate the primary shards to a valid
|
|
|
-node. After shrinking, it will swap aliases pointing to the original index
|
|
|
-into the new shrunken index. The new index will also have a new name:
|
|
|
-"shrink-<origin-index-name>". So if the original index was called "logs",
|
|
|
-then the new index will be named "shrink-logs".
|
|
|
-
|
|
|
-[[ilm-shrink-options]]
|
|
|
-.Shrink Options
|
|
|
-[options="header"]
|
|
|
-|======
|
|
|
-| Name | Required | Default | Description
|
|
|
-| `number_of_shards` | yes | - | The number of shards
|
|
|
- to shrink to. must be
|
|
|
- a factor of the number
|
|
|
- of shards in the
|
|
|
- source index.
|
|
|
-|======
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT _ilm/policy/my_policy
|
|
|
-{
|
|
|
- "policy": {
|
|
|
- "phases": {
|
|
|
- "warm": {
|
|
|
- "actions": {
|
|
|
- "shrink" : {
|
|
|
- "number_of_shards": 1
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-
|
|
|
-[[ilm-unfollow-action]]
|
|
|
-==== Unfollow
|
|
|
-
|
|
|
-[IMPORTANT]
|
|
|
-This action may be used explicitly, as shown below, but this action is also run
|
|
|
-before <<ilm-rollover-action,the Rollover action>> and <<ilm-shrink-action,the
|
|
|
-Shrink action>> as described in the documentation for those actions. This is
|
|
|
-expected and safe for non-CCR indices to run, as the steps are skipped when CCR
|
|
|
-is not in use.
|
|
|
-
|
|
|
-This action turns a {ref}/ccr-apis.html[ccr] follower index
|
|
|
-into a regular index. This can be desired when moving follower
|
|
|
-indices into the next phase. Also certain actions like shrink
|
|
|
-and rollover can then be performed safely on follower indices.
|
|
|
-
|
|
|
-This action will wait until is it safe to convert a follower index into a
|
|
|
-regular index. In particular, the following conditions must be met:
|
|
|
-
|
|
|
-* The leader index must have `index.lifecycle.indexing_complete` set to `true`.
|
|
|
-This happens automatically if the leader index is rolled over using
|
|
|
-<<ilm-rollover-action,the Rollover action>>, or may be set manually using
|
|
|
-the <<indices-update-settings,Index Settings API>>.
|
|
|
-* All operations performed on the leader index must have been replicated to the
|
|
|
-follower index. This ensures that no operations will be lost when the index is
|
|
|
-converted into a regular index.
|
|
|
-
|
|
|
-If the unfollow action encounters a follower index then
|
|
|
-the following operations will be performed on it:
|
|
|
-
|
|
|
-* Pauses indexing following for the follower index.
|
|
|
-* Closes the follower index.
|
|
|
-* Unfollows the follower index.
|
|
|
-* Opens the follower index (which is at this point is a regular index).
|
|
|
-
|
|
|
-The unfollow action does not have any options and
|
|
|
-if it encounters a non follower index, then the
|
|
|
-unfollow action leaves that index untouched and
|
|
|
-lets the next action operate on this index.
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT _ilm/policy/my_policy
|
|
|
-{
|
|
|
- "policy": {
|
|
|
- "phases": {
|
|
|
- "hot": {
|
|
|
- "actions": {
|
|
|
- "unfollow" : {}
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-
|
|
|
-=== Full Policy
|
|
|
-
|
|
|
-With all of these actions, we can support complex management strategies for our
|
|
|
-indices. This policy will define an index that will start in the hot phase,
|
|
|
-rolling over every 50 GB or 7 days. After 30 days it enters the warm phase
|
|
|
-and increases the replicas to 2, force merges and shrinks. After 60 days
|
|
|
-it enters the cold phase and allocates to "cold" nodes, and after 90 days the
|
|
|
-index is deleted.
|
|
|
-
|
|
|
-[source,console]
|
|
|
---------------------------------------------------
|
|
|
-PUT _ilm/policy/full_policy
|
|
|
-{
|
|
|
- "policy": {
|
|
|
- "phases": {
|
|
|
- "hot": {
|
|
|
- "actions": {
|
|
|
- "rollover": {
|
|
|
- "max_age": "7d",
|
|
|
- "max_size": "50G"
|
|
|
- }
|
|
|
- }
|
|
|
- },
|
|
|
- "warm": {
|
|
|
- "min_age": "30d",
|
|
|
- "actions": {
|
|
|
- "forcemerge": {
|
|
|
- "max_num_segments": 1
|
|
|
- },
|
|
|
- "shrink": {
|
|
|
- "number_of_shards": 1
|
|
|
- },
|
|
|
- "allocate": {
|
|
|
- "number_of_replicas": 2
|
|
|
- }
|
|
|
- }
|
|
|
- },
|
|
|
- "cold": {
|
|
|
- "min_age": "60d",
|
|
|
- "actions": {
|
|
|
- "allocate": {
|
|
|
- "require": {
|
|
|
- "type": "cold"
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- },
|
|
|
- "delete": {
|
|
|
- "min_age": "90d",
|
|
|
- "actions": {
|
|
|
- "delete": {}
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|