123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172 |
- [role="xpack"]
- [[data-stream-lifecycle-settings]]
- === Data stream lifecycle settings in {es}
- [subs="attributes"]
- ++++
- <titleabbrev>Data stream lifecycle settings</titleabbrev>
- ++++
- preview::[]
- These are the settings available for configuring <<data-stream-lifecycle, data stream lifecycle>>.
- ==== Cluster level settings
- [[data-streams-lifecycle-poll-interval]]
- `data_streams.lifecycle.poll_interval`::
- (<<dynamic-cluster-setting,Dynamic>>, <<time-units, time unit value>>)
- How often {es} checks what is the next action for all data streams with a built-in lifecycle. Defaults to `5m`.
- [[cluster-lifecycle-default-rollover]]
- `cluster.lifecycle.default.rollover`::
- (<<dynamic-cluster-setting,Dynamic>>, string)
- This property accepts a key value pair formatted string and configures the conditions that would trigger a data stream
- to <<index-rollover,rollover>> when it has `lifecycle` configured. This property is an implementation detail and subject to
- change. Currently, it defaults to `max_age=auto,max_primary_shard_size=50gb,min_docs=1,max_primary_shard_docs=200000000`,
- this means that your data stream will rollover if any of the following conditions are met:
- * Either any primary shard reaches the size of 50GB,
- * or any primary shard contains 200.000.000 documents
- * or the index reaches a certain age which depends on the retention time of your data stream,
- * **and** has at least one document.
- [[data-streams-lifecycle-target-merge-factor]]
- `data_streams.lifecycle.target.merge.policy.merge_factor`::
- (<<dynamic-cluster-setting,Dynamic>>, integer)
- Data stream lifecycle implements <<data-streams-lifecycle-how-it-works, tail merging>> by
- updating the lucene merge policy factor for the target backing index. The merge factor
- is both the number of segments that should be merged together, and the maximum number
- of segments that we expect to find on a given tier.
- This setting controls what value does <<data-stream-lifecycle, Data stream lifecycle>>
- configures on the target index. It defaults to `16`.
- The value will be visible under the `index.merge.policy.merge_factor` index setting
- on the target index.
- [[data-streams-lifecycle-target-floor-segment]]
- `data_streams.lifecycle.target.merge.policy.floor_segment`::
- (<<dynamic-cluster-setting,Dynamic>>)
- Data stream lifecycle implements <<data-streams-lifecycle-how-it-works, tail merging>> by
- updating the lucene merge policy floor segment for the target backing index. This floor
- segment size is a way to prevent indices from having a long tail of very small segments.
- This setting controls what value does <<data-stream-lifecycle, data stream lifecycle>>
- configures on the target index. It defaults to `100MB`.
- ==== Index level settings
- The following index-level settings are typically configured on the backing indices of a data stream.
- [[index-lifecycle-prefer-ilm]]
- `index.lifecycle.prefer_ilm`::
- (<<indices-update-settings,Dynamic>>, boolean)
- This setting determines which feature is managing the backing index of a data stream if, and only if, the backing index
- has an <<index-lifecycle-management,{ilm}>> ({ilm-init}) policy and the data stream has also a built-in lifecycle. When
- `true` this index is managed by {ilm-init}, when `false` the backing index is managed by the data stream lifecycle.
- Defaults to `true`.
- [[index-data-stream-lifecycle-origination-date]]
- `index.lifecycle.origination_date`::
- (<<indices-update-settings,Dynamic>>, long)
- If specified, this is the timestamp used to calculate the backing index generation age after this backing index has been
- <<index-rollover,rolled over>>. The generation age is used to determine data retention, consequently, you can use this
- setting if you create a backing index that contains older data and want to ensure that the retention period or
- other parts of the lifecycle will be applied based on the data's original timestamp and not the timestamp they got
- indexed. Specified as a Unix epoch value in milliseconds.
|