diagnose-unassigned-shards.asciidoc 8.4 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284
  1. ////
  2. [source,console]
  3. ----
  4. PUT my-index-000001?master_timeout=1s&timeout=1s
  5. {
  6. "settings": {
  7. "index.routing.allocation.include._name": "nonexistent_node",
  8. "index.routing.allocation.include._tier_preference": null
  9. }
  10. }
  11. ----
  12. ////
  13. // tag::cloud[]
  14. In order to diagnose the unassigned shards, follow the next steps:
  15. **Use {kib}**
  16. //tag::kibana-api-ex[]
  17. . Log in to the {ess-console}[{ecloud} console].
  18. +
  19. . On the **Elasticsearch Service** panel, click the name of your deployment.
  20. +
  21. NOTE: If the name of your deployment is disabled your {kib} instances might be
  22. unhealthy, in which case please contact https://support.elastic.co[Elastic Support].
  23. If your deployment doesn't include {kib}, all you need to do is
  24. {cloud}/ec-access-kibana.html[enable it first].
  25. . Open your deployment's side navigation menu (placed under the Elastic logo in the upper left corner)
  26. and go to **Dev Tools > Console**.
  27. +
  28. [role="screenshot"]
  29. image::images/kibana-console.png[{kib} Console,align="center"]
  30. . View the unassigned shards using the <<cat-shards,cat shards API>>.
  31. +
  32. [source,console]
  33. ----
  34. GET _cat/shards?v=true&h=index,shard,prirep,state,node,unassigned.reason&s=state
  35. ----
  36. +
  37. The response will look like this:
  38. +
  39. [source,console-result]
  40. ----
  41. [
  42. {
  43. "index": "my-index-000001",
  44. "shard": "0",
  45. "prirep": "p",
  46. "state": "UNASSIGNED",
  47. "node": null,
  48. "unassigned.reason": "INDEX_CREATED"
  49. }
  50. ]
  51. ----
  52. // TEST[skip:illustration purposes only]
  53. +
  54. Unassigned shards have a `state` of `UNASSIGNED`. The `prirep` value is `p` for
  55. primary shards and `r` for replicas.
  56. +
  57. The index in the example has a primary shard unassigned.
  58. . To understand why an unassigned shard is not being assigned and what action
  59. you must take to allow {es} to assign it, use the
  60. <<cluster-allocation-explain,cluster allocation explanation API>>.
  61. +
  62. [source,console]
  63. ----
  64. GET _cluster/allocation/explain
  65. {
  66. "index": "my-index-000001", <1>
  67. "shard": 0, <2>
  68. "primary": true <3>
  69. }
  70. ----
  71. // TEST[skip:illustration purposes only]
  72. +
  73. <1> The index we want to diagnose.
  74. +
  75. <2> The unassigned shard ID.
  76. +
  77. <3> Indicates that we are diagnosing a primary shard.
  78. +
  79. The response will look like this:
  80. +
  81. [source,console-result]
  82. ----
  83. {
  84. "index" : "my-index-000001",
  85. "shard" : 0,
  86. "primary" : true,
  87. "current_state" : "unassigned", <1>
  88. "unassigned_info" : {
  89. "reason" : "INDEX_CREATED", <2>
  90. "at" : "2022-01-04T18:08:16.600Z",
  91. "last_allocation_status" : "no"
  92. },
  93. "can_allocate" : "no", <3>
  94. "allocate_explanation" : "Elasticsearch isn't allowed to allocate this shard to any of the nodes in the cluster. Choose a node to which you expect this shard to be allocated, find this node in the node-by-node explanation, and address the reasons which prevent Elasticsearch from allocating this shard there.",
  95. "node_allocation_decisions" : [
  96. {
  97. "node_id" : "8qt2rY-pT6KNZB3-hGfLnw",
  98. "node_name" : "node-0",
  99. "transport_address" : "127.0.0.1:9401",
  100. "roles": ["data_content", "data_hot"],
  101. "node_attributes" : {},
  102. "node_decision" : "no", <4>
  103. "weight_ranking" : 1,
  104. "deciders" : [
  105. {
  106. "decider" : "filter", <5>
  107. "decision" : "NO",
  108. "explanation" : "node does not match index setting [index.routing.allocation.include] filters [_name:\"nonexistent_node\"]" <6>
  109. }
  110. ]
  111. }
  112. ]
  113. }
  114. ----
  115. // TEST[skip:illustration purposes only]
  116. +
  117. <1> The current state of the shard.
  118. +
  119. <2> The reason for the shard originally becoming unassigned.
  120. +
  121. <3> Whether to allocate the shard.
  122. +
  123. <4> Whether to allocate the shard to the particular node.
  124. +
  125. <5> The decider which led to the `no` decision for the node.
  126. +
  127. <6> An explanation as to why the decider returned a `no` decision, with a helpful hint pointing to the setting that led to the decision.
  128. . The explanation in our case indicates the index allocation configurations are not correct.
  129. To review your allocation settings, use the <<indices-get-settings,get index
  130. settings>> and <<cluster-get-settings,cluster get settings>> APIs.
  131. +
  132. [source,console]
  133. ----
  134. GET my-index-000001/_settings?flat_settings=true&include_defaults=true
  135. GET _cluster/settings?flat_settings=true&include_defaults=true
  136. ----
  137. // TEST[s/^/PUT my-index-000001\n/]
  138. . Change the settings using the <<indices-update-settings,update index
  139. settings>> and <<cluster-update-settings,cluster update settings>> APIs to the
  140. correct values in order to allow the index to be allocated.
  141. For more guidance on fixing the most common causes for unassinged shards please follow
  142. <<fix-red-yellow-cluster-status, this guide>> or contact https://support.elastic.co[Elastic Support].
  143. //end::kibana-api-ex[]
  144. // end::cloud[]
  145. // tag::self-managed[]
  146. In order to diagnose the unassigned shards follow the next steps:
  147. . View the unassigned shards using the <<cat-shards,cat shards API>>.
  148. +
  149. [source,console]
  150. ----
  151. GET _cat/shards?v=true&h=index,shard,prirep,state,node,unassigned.reason&s=state
  152. ----
  153. +
  154. The response will look like this:
  155. +
  156. [source,console-result]
  157. ----
  158. [
  159. {
  160. "index": "my-index-000001",
  161. "shard": "0",
  162. "prirep": "p",
  163. "state": "UNASSIGNED",
  164. "node": null,
  165. "unassigned.reason": "INDEX_CREATED"
  166. }
  167. ]
  168. ----
  169. // TEST[skip:illustration purposes only]
  170. +
  171. Unassigned shards have a `state` of `UNASSIGNED`. The `prirep` value is `p` for
  172. primary shards and `r` for replicas.
  173. +
  174. The index in the example has a primary shard unassigned.
  175. . To understand why an unassigned shard is not being assigned and what action
  176. you must take to allow {es} to assign it, use the
  177. <<cluster-allocation-explain,cluster allocation explanation API>>.
  178. +
  179. [source,console]
  180. ----
  181. GET _cluster/allocation/explain
  182. {
  183. "index": "my-index-000001", <1>
  184. "shard": 0, <2>
  185. "primary": true <3>
  186. }
  187. ----
  188. // TEST[skip:illustration purposes only]
  189. +
  190. <1> The index we want to diagnose.
  191. +
  192. <2> The unassigned shard ID.
  193. +
  194. <3> Indicates that we are diagnosing a primary shard.
  195. +
  196. The response will look like this:
  197. +
  198. [source,console-result]
  199. ----
  200. {
  201. "index" : "my-index-000001",
  202. "shard" : 0,
  203. "primary" : true,
  204. "current_state" : "unassigned", <1>
  205. "unassigned_info" : {
  206. "reason" : "INDEX_CREATED", <2>
  207. "at" : "2022-01-04T18:08:16.600Z",
  208. "last_allocation_status" : "no"
  209. },
  210. "can_allocate" : "no", <3>
  211. "allocate_explanation" : "Elasticsearch isn't allowed to allocate this shard to any of the nodes in the cluster. Choose a node to which you expect this shard to be allocated, find this node in the node-by-node explanation, and address the reasons which prevent Elasticsearch from allocating this shard there.",
  212. "node_allocation_decisions" : [
  213. {
  214. "node_id" : "8qt2rY-pT6KNZB3-hGfLnw",
  215. "node_name" : "node-0",
  216. "transport_address" : "127.0.0.1:9401",
  217. "roles": ["data_content", "data_hot"]
  218. "node_attributes" : {},
  219. "node_decision" : "no", <4>
  220. "weight_ranking" : 1,
  221. "deciders" : [
  222. {
  223. "decider" : "filter", <5>
  224. "decision" : "NO",
  225. "explanation" : "node does not match index setting [index.routing.allocation.include] filters [_name:\"nonexistent_node\"]" <6>
  226. }
  227. ]
  228. }
  229. ]
  230. }
  231. ----
  232. // TEST[skip:illustration purposes only]
  233. +
  234. <1> The current state of the shard.
  235. +
  236. <2> The reason for the shard originally becoming unassigned.
  237. +
  238. <3> Whether to allocate the shard.
  239. +
  240. <4> Whether to allocate the shard to the particular node.
  241. +
  242. <5> The decider which led to the `no` decision for the node.
  243. +
  244. <6> An explanation as to why the decider returned a `no` decision, with a helpful hint pointing to the setting that led to the decision.
  245. . The explanation in our case indicates the index allocation configurations are not correct.
  246. To review your allocation settings, use the <<indices-get-settings,get index
  247. settings>> and <<cluster-get-settings,cluster get settings>> APIs.
  248. +
  249. [source,console]
  250. ----
  251. GET my-index-000001/_settings?flat_settings=true&include_defaults=true
  252. GET _cluster/settings?flat_settings=true&include_defaults=true
  253. ----
  254. // TEST[s/^/PUT my-index-000001\n/]
  255. . Change the settings using the <<indices-update-settings,update index
  256. settings>> and <<cluster-update-settings,cluster update settings>> APIs to the
  257. correct values in order to allow the index to be allocated.
  258. For more guidance on fixing the most common causes for unassinged shards please follow
  259. <<fix-red-yellow-cluster-status, this guide>>.
  260. // end::self-managed[]