cluster.asciidoc 9.5 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249
  1. [[modules-cluster]]
  2. == Cluster
  3. [float]
  4. [[shards-allocation]]
  5. === Shards Allocation
  6. Shards allocation is the process of allocating shards to nodes. This can
  7. happen during initial recovery, replica allocation, rebalancing, or
  8. handling nodes being added or removed.
  9. The following settings may be used:
  10. `cluster.routing.allocation.allow_rebalance`::
  11. Allow to control when rebalancing will happen based on the total
  12. state of all the indices shards in the cluster. `always`,
  13. `indices_primaries_active`, and `indices_all_active` are allowed,
  14. defaulting to `indices_all_active` to reduce chatter during
  15. initial recovery.
  16. `cluster.routing.allocation.cluster_concurrent_rebalance`::
  17. Allow to control how many concurrent rebalancing of shards are
  18. allowed cluster wide, and default it to `2`.
  19. `cluster.routing.allocation.node_initial_primaries_recoveries`::
  20. Allow to control specifically the number of initial recoveries
  21. of primaries that are allowed per node. Since most times local
  22. gateway is used, those should be fast and we can handle more of
  23. those per node without creating load.
  24. `cluster.routing.allocation.node_concurrent_recoveries`::
  25. How many concurrent recoveries are allowed to happen on a node.
  26. Defaults to `2`.
  27. `cluster.routing.allocation.enable`::
  28. Controls shard allocation for all indices, by allowing specific
  29. kinds of shard to be allocated.
  30. +
  31. --
  32. Can be set to:
  33. * `all` - (default) Allows shard allocation for all kinds of shards.
  34. * `primaries` - Allows shard allocation only for primary shards.
  35. * `new_primaries` - Allows shard allocation only for primary shards for new indices.
  36. * `none` - No shard allocations of any kind are allowed for all indices.
  37. --
  38. `cluster.routing.rebalance.enable`::
  39. Controls shard rebalance for all indices, by allowing specific
  40. kinds of shard to be rebalanced.
  41. +
  42. --
  43. Can be set to:
  44. * `all` - (default) Allows shard balancing for all kinds of shards.
  45. * `primaries` - Allows shard balancing only for primary shards.
  46. * `replicas` - Allows shard balancing only for replica shards.
  47. * `none` - No shard balancing of any kind are allowed for all indices.
  48. --
  49. `cluster.routing.allocation.same_shard.host`::
  50. Allows to perform a check to prevent allocation of multiple instances
  51. of the same shard on a single host, based on host name and host address.
  52. Defaults to `false`, meaning that no check is performed by default. This
  53. setting only applies if multiple nodes are started on the same machine.
  54. `indices.recovery.concurrent_streams`::
  55. The number of streams to open (on a *node* level) to recover a
  56. shard from a peer shard. Defaults to `3`.
  57. [float]
  58. [[allocation-awareness]]
  59. === Shard Allocation Awareness
  60. Cluster allocation awareness allows to configure shard and replicas
  61. allocation across generic attributes associated the nodes. Lets explain
  62. it through an example:
  63. Assume we have several racks. When we start a node, we can configure an
  64. attribute called `rack_id` (any attribute name works), for example, here
  65. is a sample config:
  66. ----------------------
  67. node.rack_id: rack_one
  68. ----------------------
  69. The above sets an attribute called `rack_id` for the relevant node with
  70. a value of `rack_one`. Now, we need to configure the `rack_id` attribute
  71. as one of the awareness allocation attributes (set it on *all* (master
  72. eligible) nodes config):
  73. --------------------------------------------------------
  74. cluster.routing.allocation.awareness.attributes: rack_id
  75. --------------------------------------------------------
  76. The above will mean that the `rack_id` attribute will be used to do
  77. awareness based allocation of shard and its replicas. For example, lets
  78. say we start 2 nodes with `node.rack_id` set to `rack_one`, and deploy a
  79. single index with 5 shards and 1 replica. The index will be fully
  80. deployed on the current nodes (5 shards and 1 replica each, total of 10
  81. shards).
  82. Now, if we start two more nodes, with `node.rack_id` set to `rack_two`,
  83. shards will relocate to even the number of shards across the nodes, but,
  84. a shard and its replica will not be allocated in the same `rack_id`
  85. value.
  86. The awareness attributes can hold several values, for example:
  87. -------------------------------------------------------------
  88. cluster.routing.allocation.awareness.attributes: rack_id,zone
  89. -------------------------------------------------------------
  90. *NOTE*: When using awareness attributes, shards will not be allocated to
  91. nodes that don't have values set for those attributes.
  92. [float]
  93. [[forced-awareness]]
  94. === Forced Awareness
  95. Sometimes, we know in advance the number of values an awareness
  96. attribute can have, and more over, we would like never to have more
  97. replicas than needed allocated on a specific group of nodes with the
  98. same awareness attribute value. For that, we can force awareness on
  99. specific attributes.
  100. For example, lets say we have an awareness attribute called `zone`, and
  101. we know we are going to have two zones, `zone1` and `zone2`. Here is how
  102. we can force awareness on a node:
  103. [source,js]
  104. -------------------------------------------------------------------
  105. cluster.routing.allocation.awareness.force.zone.values: zone1,zone2
  106. cluster.routing.allocation.awareness.attributes: zone
  107. -------------------------------------------------------------------
  108. Now, lets say we start 2 nodes with `node.zone` set to `zone1` and
  109. create an index with 5 shards and 1 replica. The index will be created,
  110. but only 5 shards will be allocated (with no replicas). Only when we
  111. start more shards with `node.zone` set to `zone2` will the replicas be
  112. allocated.
  113. [float]
  114. ==== Automatic Preference When Searching / GETing
  115. When executing a search, or doing a get, the node receiving the request
  116. will prefer to execute the request on shards that exists on nodes that
  117. have the same attribute values as the executing node. This only happens
  118. when the `cluster.routing.allocation.awareness.attributes` setting has
  119. been set to a value.
  120. [float]
  121. ==== Realtime Settings Update
  122. The settings can be updated using the <<cluster-update-settings,cluster update settings API>> on a live cluster.
  123. [float]
  124. [[allocation-filtering]]
  125. === Shard Allocation Filtering
  126. Allow to control allocation of indices on nodes based on include/exclude
  127. filters. The filters can be set both on the index level and on the
  128. cluster level. Lets start with an example of setting it on the cluster
  129. level:
  130. Lets say we have 4 nodes, each has specific attribute called `tag`
  131. associated with it (the name of the attribute can be any name). Each
  132. node has a specific value associated with `tag`. Node 1 has a setting
  133. `node.tag: value1`, Node 2 a setting of `node.tag: value2`, and so on.
  134. We can create an index that will only deploy on nodes that have `tag`
  135. set to `value1` and `value2` by setting
  136. `index.routing.allocation.include.tag` to `value1,value2`. For example:
  137. [source,js]
  138. --------------------------------------------------
  139. curl -XPUT localhost:9200/test/_settings -d '{
  140. "index.routing.allocation.include.tag" : "value1,value2"
  141. }'
  142. --------------------------------------------------
  143. On the other hand, we can create an index that will be deployed on all
  144. nodes except for nodes with a `tag` of value `value3` by setting
  145. `index.routing.allocation.exclude.tag` to `value3`. For example:
  146. [source,js]
  147. --------------------------------------------------
  148. curl -XPUT localhost:9200/test/_settings -d '{
  149. "index.routing.allocation.exclude.tag" : "value3"
  150. }'
  151. --------------------------------------------------
  152. `index.routing.allocation.require.*` can be used to
  153. specify a number of rules, all of which MUST match in order for a shard
  154. to be allocated to a node. This is in contrast to `include` which will
  155. include a node if ANY rule matches.
  156. The `include`, `exclude` and `require` values can have generic simple
  157. matching wildcards, for example, `value1*`. A special attribute name
  158. called `_ip` can be used to match on node ip values. In addition `_host`
  159. attribute can be used to match on either the node's hostname or its ip
  160. address. Similarly `_name` and `_id` attributes can be used to match on
  161. node name and node id accordingly.
  162. Obviously a node can have several attributes associated with it, and
  163. both the attribute name and value are controlled in the setting. For
  164. example, here is a sample of several node configurations:
  165. [source,js]
  166. --------------------------------------------------
  167. node.group1: group1_value1
  168. node.group2: group2_value4
  169. --------------------------------------------------
  170. In the same manner, `include`, `exclude` and `require` can work against
  171. several attributes, for example:
  172. [source,js]
  173. --------------------------------------------------
  174. curl -XPUT localhost:9200/test/_settings -d '{
  175. "index.routing.allocation.include.group1" : "xxx"
  176. "index.routing.allocation.include.group2" : "yyy",
  177. "index.routing.allocation.exclude.group3" : "zzz",
  178. "index.routing.allocation.require.group4" : "aaa"
  179. }'
  180. --------------------------------------------------
  181. The provided settings can also be updated in real time using the update
  182. settings API, allowing to "move" indices (shards) around in realtime.
  183. Cluster wide filtering can also be defined, and be updated in real time
  184. using the cluster update settings API. This setting can come in handy
  185. for things like decommissioning nodes (even if the replica count is set
  186. to 0). Here is a sample of how to decommission a node based on `_ip`
  187. address:
  188. [source,js]
  189. --------------------------------------------------
  190. curl -XPUT localhost:9200/_cluster/settings -d '{
  191. "transient" : {
  192. "cluster.routing.allocation.exclude._ip" : "10.0.0.1"
  193. }
  194. }'
  195. --------------------------------------------------