Browse Source

(Doc) ILM Force Merge not on HDD and happens on hosting node not current phase tier (#130280)

* Update ilm-forcemerge.md

Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com>
Co-authored-by: Niels Bauman <33722607+nielsbauman@users.noreply.github.com>
Stef Nestor 2 months ago
parent
commit
b4c2c2e42c
1 changed files with 11 additions and 10 deletions
  1. 11 10
      docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md

+ 11 - 10
docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md

@@ -7,29 +7,30 @@ mapped_pages:
 
 Phases allowed: hot, warm.
 
-[Force merges](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-forcemerge) the index into the specified maximum number of [segments](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-segments).
-
-::::{note}
-Shards that are relocating during a `forcemerge` will not be merged.
-::::
-
+[Force merges](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-forcemerge) the index into the specified maximum number of [segments](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-segments). This operation is performed on a best effort basis. For example, shards that are relocating during a `forcemerge` will not be merged.
 
 To use the `forcemerge` action in the `hot` phase, the `rollover` action **must** be present. If no rollover action is configured, {{ilm-init}} will reject the policy.
 
 :::::{admonition} Performance considerations
 :name: ilm-forcemerge-performance
 
-Force merge is a resource-intensive operation. If too many force merges are triggered at once, it can negatively impact your cluster. This can happen when you apply an {{ilm-init}} policy that includes a force merge action to existing indices. If they meet the `min_age` criteria, they can immediately proceed through multiple phases. You can prevent this by increasing the `min_age` or setting `index.lifecycle.origination_date` to change how the index age is calculated.
+Force merge is a resource-intensive operation. If too many force merges are triggered at once, it can negatively impact your cluster. For example, this can happen when you 
+* modify an existing {{ilm-init}} policy's phase `min_age`, causing indices to trigger the force merge at a faster rate.
+* apply an {{ilm-init}} policy that includes a force merge action to existing indices. If the indices meet the `min_age` criteria, they can immediately proceed through multiple actions. You can prevent this by increasing the `min_age` or setting `index.lifecycle.origination_date` to change how the index age is calculated.
+* run the [{{ilm-init}} Move Step API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ilm-move-to-step) against multiple indices.
 
 If you experience a force merge task queue backlog, you might need to increase the size of the force merge threadpool so indices can be force merged in parallel. To do this, configure the `thread_pool.force_merge.size` [cluster setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-get-settings).
 
 ::::{important}
-This can have cascading performance impacts. Monitor cluster performance and increment the size of the thread pool slowly to reduce the backlog.
+Note that `thread_pool.force_merge.size` is an advanced setting. Adjusting it can cause cascading performance impacts. Monitor cluster performance and increment the size of the thread pool slowly to reduce the backlog.
 ::::
 
+Force merging will be performed by the node hosting the shard. Usually, the [node's role](docs-content://deploy-manage/distributed-architecture/clusters-nodes-shards/node-roles.md#set-node-roles) matches the [data tier](docs-content://manage-data/lifecycle/data-tiers.md) of the {{ilm-init}} phase that the index is in. One of the exceptions is when you have manually disabled [ILM Migrate](https://www.elastic.co/docs/reference/elasticsearch/index-lifecycle-actions/ilm-migrate) and have specified custom allocations using [ILM allocate](https://www.elastic.co/docs/reference/elasticsearch/index-lifecycle-actions/ilm-allocate). The other exception is searchable snapshots; force merges for [{{ilm-init}} Searchable Snapshots](./ilm-searchable-snapshot.md) using `force_merge_index` are performed in the phase that the index is in **prior** to the `searchable_snapshot` action. You may want to explicitly choose in which data tier the force merge should occur, for example:
+* A force merge in the `hot` phase will use hot nodes. Merges may be faster on this potentially higher performance hardware but may have the tradeoff of impacting ingestion. 
+* A force merge in the `warm` phase will use warm nodes. Merges may take longer to perform on potentially lower performance hardware but will avoid impacting ingestion in the `hot` tier.
+*  [{{ilm-init}} Searchable Snapshot](./ilm-searchable-snapshot.md) performance is dependant upon the shard having been force merged, so by default this ILM action will enable `force_merge_index`. This will trigger force merges in the preceding node data tier for `cold` and `frozen` phases.
 
-Force merging will be performed by the nodes within the current phase of the index. A forcemerge in the `hot` phase will use hot nodes with potentially faster nodes, while impacting ingestion more. A forcemerge in the `warm` phase will use warm nodes and potentially take longer to perform, but without impacting ingestion in the `hot` tier.
-
+We recommend that merges be targetted against SSD and not HDD disks.
 :::::