ilm-index-lifecycle.asciidoc 4.7 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113
  1. [role="xpack"]
  2. [testenv="basic"]
  3. [[ilm-index-lifecycle]]
  4. === Index lifecycle
  5. ++++
  6. <titleabbrev>Index lifecycle</titleabbrev>
  7. ++++
  8. {ilm-init} defines five index lifecycle _phases_:
  9. * **Hot**: The index is actively being updated and queried.
  10. * **Warm**: The index is no longer being updated but is still being queried.
  11. * **Cold**: The index is no longer being updated and is queried infrequently. The information still
  12. needs to be searchable, but it's okay if those queries are slower.
  13. * **Frozen**: The index is no longer being updated and is queried rarely. The information still
  14. needs to be searchable, but it's okay if those queries are extremely slow.
  15. * **Delete**: The index is no longer needed and can safely be removed.
  16. An index's _lifecycle policy_ specifies which phases
  17. are applicable, what actions are performed in each phase,
  18. and when it transitions between phases.
  19. You can manually apply a lifecycle policy when you create an index.
  20. For time series indices, you need to associate the lifecycle policy with
  21. the index template used to create new indices in the series.
  22. When an index rolls over, a manually-applied policy isn't automatically applied to the new index.
  23. If you use {es}'s security features, {ilm-init} performs operations as the user
  24. who last updated the policy. {ilm-init} only has the
  25. <<defining-roles,roles>> assigned to the user at the time of the last policy
  26. update.
  27. [discrete]
  28. [[ilm-phase-transitions]]
  29. === Phase transitions
  30. {ilm-init} moves indices through the lifecycle according to their age.
  31. To control the timing of these transitions, you set a _minimum age_ for each phase. For an index to
  32. move to the next phase, all actions in the current phase must be complete and the index must be
  33. older than the minimum age of the next phase. Configured minimum ages must increase between
  34. subsequent phases, for example, a "warm" phase with a minimum age of 10 days can only be followed by
  35. a "cold" phase with a minimum age either unset, or >= 10 days.
  36. The minimum age defaults to zero, which causes {ilm-init} to move indices to the next phase
  37. as soon as all actions in the current phase complete.
  38. If an index has unallocated shards and the <<cluster-health,cluster health status>> is yellow,
  39. the index can still transition to the next phase according to its {ilm} policy.
  40. However, because {es} can only perform certain clean up tasks on a green
  41. cluster, there might be unexpected side effects.
  42. To avoid increased disk usage and reliability issues,
  43. address any cluster health problems in a timely fashion.
  44. [discrete]
  45. [[ilm-phase-execution]]
  46. === Phase execution
  47. {ilm-init} controls the order in which the actions in a phase are executed and
  48. what _steps_ are executed to perform the necessary index operations for each action.
  49. When an index enters a phase, {ilm-init} caches the phase definition in the index metadata.
  50. This ensures that policy updates don't put the index into a state where it can never exit the phase.
  51. If changes can be safely applied, {ilm-init} updates the cached phase definition.
  52. If they cannot, phase execution continues using the cached definition.
  53. {ilm-init} runs periodically, checks to see if an index meets policy criteria,
  54. and executes whatever steps are needed.
  55. To avoid race conditions, {ilm-init} might need to run more than once to execute all of the steps
  56. required to complete an action.
  57. For example, if {ilm-init} determines that an index has met the rollover criteria,
  58. it begins executing the steps required to complete the rollover action.
  59. If it reaches a point where it is not safe to advance to the next step, execution stops.
  60. The next time {ilm-init} runs, {ilm-init} picks up execution where it left off.
  61. This means that even if `indices.lifecycle.poll_interval` is set to 10 minutes and an index meets
  62. the rollover criteria, it could be 20 minutes before the rollover is complete.
  63. [discrete]
  64. [[ilm-phase-actions]]
  65. === Phase actions
  66. {ilm-init} supports the following actions in each phase. {ilm-init} executes the
  67. actions in the order listed.
  68. * Hot
  69. - <<ilm-set-priority,Set Priority>>
  70. - <<ilm-unfollow,Unfollow>>
  71. - <<ilm-rollover,Rollover>>
  72. - <<ilm-readonly,Read-Only>>
  73. - <<ilm-shrink,Shrink>>
  74. - <<ilm-forcemerge,Force Merge>>
  75. - <<ilm-searchable-snapshot, Searchable Snapshot>>
  76. * Warm
  77. - <<ilm-set-priority,Set Priority>>
  78. - <<ilm-unfollow,Unfollow>>
  79. - <<ilm-readonly,Read-Only>>
  80. - <<ilm-allocate,Allocate>>
  81. - <<ilm-migrate,Migrate>>
  82. - <<ilm-shrink,Shrink>>
  83. - <<ilm-forcemerge,Force Merge>>
  84. * Cold
  85. - <<ilm-set-priority,Set Priority>>
  86. - <<ilm-unfollow,Unfollow>>
  87. - <<ilm-readonly,Read-Only>>
  88. - <<ilm-searchable-snapshot, Searchable Snapshot>>
  89. - <<ilm-allocate,Allocate>>
  90. - <<ilm-migrate,Migrate>>
  91. * Frozen
  92. - <<ilm-searchable-snapshot, Searchable Snapshot>>
  93. * Delete
  94. - <<ilm-wait-for-snapshot,Wait For Snapshot>>
  95. - <<ilm-delete,Delete>>