recovery.asciidoc 11 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463
  1. [[indices-recovery]]
  2. === Index recovery API
  3. ++++
  4. <titleabbrev>Index recovery</titleabbrev>
  5. ++++
  6. Returns information about ongoing and completed shard recoveries for one or more
  7. indices. For data streams, the API returns information for the stream's backing
  8. indices.
  9. [source,console]
  10. ----
  11. GET /my-index-000001/_recovery
  12. ----
  13. // TEST[setup:my_index]
  14. [[index-recovery-api-request]]
  15. ==== {api-request-title}
  16. `GET /<target>/_recovery`
  17. `GET /_recovery`
  18. [[index-recovery-api-prereqs]]
  19. ==== {api-prereq-title}
  20. * If the {es} {security-features} are enabled, you must have the `monitor` or
  21. `manage` <<privileges-list-indices,index privilege>> for the target data stream,
  22. index, or index alias.
  23. [[index-recovery-api-desc]]
  24. ==== {api-description-title}
  25. Use the index recovery API
  26. to get information about ongoing and completed shard recoveries.
  27. // tag::shard-recovery-desc[]
  28. Shard recovery is the process
  29. of syncing a replica shard from a primary shard.
  30. Upon completion,
  31. the replica shard is available for search.
  32. Recovery automatically occurs during the following processes:
  33. * Node startup or failure. This type of recovery is called a local store
  34. recovery.
  35. * <<glossary-replica-shard,Primary shard replication>>.
  36. * Relocation of a shard to a different node in the same cluster.
  37. * <<snapshots-restore-snapshot,Snapshot restoration>>.
  38. // end::shard-recovery-desc[]
  39. [[index-recovery-api-path-params]]
  40. ==== {api-path-parms-title}
  41. `<target>`::
  42. (Optional, string)
  43. Comma-separated list of data streams, indices, and index aliases used to limit
  44. the request. Wildcard expressions (`*`) are supported.
  45. +
  46. To target all data streams and indices in a cluster, omit this parameter or use
  47. `_all` or `*`.
  48. [[index-recovery-api-query-params]]
  49. ==== {api-query-parms-title}
  50. include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=active-only]
  51. include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=detailed]
  52. include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=index-query-parm]
  53. [[index-recovery-api-response-body]]
  54. ==== {api-response-body-title}
  55. `id`::
  56. (Integer)
  57. ID of the shard.
  58. `type`::
  59. +
  60. --
  61. (String)
  62. Recovery type.
  63. Returned values include:
  64. `STORE`::
  65. The recovery is related to
  66. a node startup or failure.
  67. This type of recovery is called a local store recovery.
  68. `SNAPSHOT`::
  69. The recovery is related to
  70. a <<snapshots-restore-snapshot,snapshot restoration>>.
  71. `REPLICA`::
  72. The recovery is related to
  73. a <<glossary-replica-shard,primary shard replication>>.
  74. `RELOCATING`::
  75. The recovery is related to
  76. the relocation of a shard
  77. to a different node in the same cluster.
  78. --
  79. `STAGE`::
  80. +
  81. --
  82. (String)
  83. Recovery stage.
  84. Returned values include:
  85. `DONE`::
  86. Complete.
  87. `FINALIZE`::
  88. Cleanup.
  89. `INDEX`::
  90. Reading index metadata and copying bytes from source to destination.
  91. `INIT`::
  92. Recovery has not started.
  93. `START`::
  94. Starting the recovery process; opening the index for use.
  95. `TRANSLOG`::
  96. Replaying transaction log .
  97. --
  98. `primary`::
  99. (Boolean)
  100. If `true`,
  101. the shard is a primary shard.
  102. `start_time`::
  103. (String)
  104. Timestamp of recovery start.
  105. `stop_time`::
  106. (String)
  107. Timestamp of recovery finish.
  108. `total_time_in_millis`::
  109. (String)
  110. Total time to recover shard in milliseconds.
  111. `source`::
  112. +
  113. --
  114. (Object)
  115. Recovery source.
  116. This can include:
  117. * A repository description if recovery is from a snapshot
  118. * A description of source node
  119. --
  120. `target`::
  121. (Object)
  122. Destination node.
  123. `index`::
  124. (Object)
  125. Statistics about physical index recovery.
  126. `translog`::
  127. (Object)
  128. Statistics about translog recovery.
  129. `start`::
  130. (Object)
  131. Statistics about time to open and start the index.
  132. [[index-recovery-api-example]]
  133. ==== {api-examples-title}
  134. [[index-recovery-api-multi-ex]]
  135. ===== Get recovery information for several data streams and indices
  136. [source,console]
  137. --------------------------------------------------
  138. GET index1,index2/_recovery?human
  139. --------------------------------------------------
  140. // TEST[s/^/PUT index1\nPUT index2\n/]
  141. [[index-recovery-api-all-ex]]
  142. ===== Get segment information for all data streams and indices in a cluster
  143. //////////////////////////
  144. Here we create a repository and snapshot index1 in
  145. order to restore it right after and prints out the
  146. index recovery result.
  147. [source,console]
  148. --------------------------------------------------
  149. # create the index
  150. PUT index1
  151. {"settings": {"index.number_of_shards": 1}}
  152. # create the repository
  153. PUT /_snapshot/my_repository
  154. {"type": "fs","settings": {"location": "recovery_asciidoc" }}
  155. # snapshot the index
  156. PUT /_snapshot/my_repository/snap_1?wait_for_completion=true
  157. {"indices": "index1"}
  158. # delete the index
  159. DELETE index1
  160. # and restore the snapshot
  161. POST /_snapshot/my_repository/snap_1/_restore?wait_for_completion=true
  162. --------------------------------------------------
  163. [source,console-result]
  164. --------------------------------------------------
  165. {
  166. "snapshot": {
  167. "snapshot": "snap_1",
  168. "indices": [
  169. "index1"
  170. ],
  171. "shards": {
  172. "total": 1,
  173. "failed": 0,
  174. "successful": 1
  175. }
  176. }
  177. }
  178. --------------------------------------------------
  179. //////////////////////////
  180. [source,console]
  181. --------------------------------------------------
  182. GET /_recovery?human
  183. --------------------------------------------------
  184. // TEST[continued]
  185. The API returns the following response:
  186. [source,console-result]
  187. --------------------------------------------------
  188. {
  189. "index1" : {
  190. "shards" : [ {
  191. "id" : 0,
  192. "type" : "SNAPSHOT",
  193. "stage" : "INDEX",
  194. "primary" : true,
  195. "start_time" : "2014-02-24T12:15:59.716",
  196. "start_time_in_millis": 1393244159716,
  197. "stop_time" : "0s",
  198. "stop_time_in_millis" : 0,
  199. "total_time" : "2.9m",
  200. "total_time_in_millis" : 175576,
  201. "source" : {
  202. "repository" : "my_repository",
  203. "snapshot" : "my_snapshot",
  204. "index" : "index1",
  205. "version" : "{version}",
  206. "restoreUUID": "PDh1ZAOaRbiGIVtCvZOMww"
  207. },
  208. "target" : {
  209. "id" : "ryqJ5lO5S4-lSFbGntkEkg",
  210. "host" : "my.fqdn",
  211. "transport_address" : "my.fqdn",
  212. "ip" : "10.0.1.7",
  213. "name" : "my_es_node"
  214. },
  215. "index" : {
  216. "size" : {
  217. "total" : "75.4mb",
  218. "total_in_bytes" : 79063092,
  219. "reused" : "0b",
  220. "reused_in_bytes" : 0,
  221. "recovered" : "65.7mb",
  222. "recovered_in_bytes" : 68891939,
  223. "percent" : "87.1%"
  224. },
  225. "files" : {
  226. "total" : 73,
  227. "reused" : 0,
  228. "recovered" : 69,
  229. "percent" : "94.5%"
  230. },
  231. "total_time" : "0s",
  232. "total_time_in_millis" : 0,
  233. "source_throttle_time" : "0s",
  234. "source_throttle_time_in_millis" : 0,
  235. "target_throttle_time" : "0s",
  236. "target_throttle_time_in_millis" : 0
  237. },
  238. "translog" : {
  239. "recovered" : 0,
  240. "total" : 0,
  241. "percent" : "100.0%",
  242. "total_on_start" : 0,
  243. "total_time" : "0s",
  244. "total_time_in_millis" : 0,
  245. },
  246. "verify_index" : {
  247. "check_index_time" : "0s",
  248. "check_index_time_in_millis" : 0,
  249. "total_time" : "0s",
  250. "total_time_in_millis" : 0
  251. }
  252. } ]
  253. }
  254. }
  255. --------------------------------------------------
  256. // TESTRESPONSE[s/: (\-)?[0-9]+/: $body.$_path/]
  257. // TESTRESPONSE[s/: "[^"]*"/: $body.$_path/]
  258. ////
  259. The TESTRESPONSE above replace all the fields values by the expected ones in the test,
  260. because we don't really care about the field values but we want to check the fields names.
  261. ////
  262. This response includes information
  263. about a single index recovering a single shard.
  264. The source of the recovery is a snapshot repository
  265. and the target of the recovery is the `my_es_node` node.
  266. The response also includes the number
  267. and percentage of files and bytes recovered.
  268. [[index-recovery-api-detailed-ex]]
  269. ===== Get detailed recovery information
  270. To get a list of physical files in recovery,
  271. set the `detailed` query parameter to `true`.
  272. [source,console]
  273. --------------------------------------------------
  274. GET _recovery?human&detailed=true
  275. --------------------------------------------------
  276. // TEST[s/^/PUT index1\n{"settings": {"index.number_of_shards": 1}}\n/]
  277. The API returns the following response:
  278. [source,console-result]
  279. --------------------------------------------------
  280. {
  281. "index1" : {
  282. "shards" : [ {
  283. "id" : 0,
  284. "type" : "STORE",
  285. "stage" : "DONE",
  286. "primary" : true,
  287. "start_time" : "2014-02-24T12:38:06.349",
  288. "start_time_in_millis" : "1393245486349",
  289. "stop_time" : "2014-02-24T12:38:08.464",
  290. "stop_time_in_millis" : "1393245488464",
  291. "total_time" : "2.1s",
  292. "total_time_in_millis" : 2115,
  293. "source" : {
  294. "id" : "RGMdRc-yQWWKIBM4DGvwqQ",
  295. "host" : "my.fqdn",
  296. "transport_address" : "my.fqdn",
  297. "ip" : "10.0.1.7",
  298. "name" : "my_es_node"
  299. },
  300. "target" : {
  301. "id" : "RGMdRc-yQWWKIBM4DGvwqQ",
  302. "host" : "my.fqdn",
  303. "transport_address" : "my.fqdn",
  304. "ip" : "10.0.1.7",
  305. "name" : "my_es_node"
  306. },
  307. "index" : {
  308. "size" : {
  309. "total" : "24.7mb",
  310. "total_in_bytes" : 26001617,
  311. "reused" : "24.7mb",
  312. "reused_in_bytes" : 26001617,
  313. "recovered" : "0b",
  314. "recovered_in_bytes" : 0,
  315. "percent" : "100.0%"
  316. },
  317. "files" : {
  318. "total" : 26,
  319. "reused" : 26,
  320. "recovered" : 0,
  321. "percent" : "100.0%",
  322. "details" : [ {
  323. "name" : "segments.gen",
  324. "length" : 20,
  325. "recovered" : 20
  326. }, {
  327. "name" : "_0.cfs",
  328. "length" : 135306,
  329. "recovered" : 135306
  330. }, {
  331. "name" : "segments_2",
  332. "length" : 251,
  333. "recovered" : 251
  334. }
  335. ]
  336. },
  337. "total_time" : "2ms",
  338. "total_time_in_millis" : 2,
  339. "source_throttle_time" : "0s",
  340. "source_throttle_time_in_millis" : 0,
  341. "target_throttle_time" : "0s",
  342. "target_throttle_time_in_millis" : 0
  343. },
  344. "translog" : {
  345. "recovered" : 71,
  346. "total" : 0,
  347. "percent" : "100.0%",
  348. "total_on_start" : 0,
  349. "total_time" : "2.0s",
  350. "total_time_in_millis" : 2025
  351. },
  352. "verify_index" : {
  353. "check_index_time" : 0,
  354. "check_index_time_in_millis" : 0,
  355. "total_time" : "88ms",
  356. "total_time_in_millis" : 88
  357. }
  358. } ]
  359. }
  360. }
  361. --------------------------------------------------
  362. // TESTRESPONSE[s/"source" : \{[^}]*\}/"source" : $body.$_path/]
  363. // TESTRESPONSE[s/"details" : \[[^\]]*\]/"details" : $body.$_path/]
  364. // TESTRESPONSE[s/: (\-)?[0-9]+/: $body.$_path/]
  365. // TESTRESPONSE[s/: "[^"]*"/: $body.$_path/]
  366. ////
  367. The TESTRESPONSE above replace all the fields values by the expected ones in the test,
  368. because we don't really care about the field values but we want to check the fields names.
  369. It also removes the "details" part which is important in this doc but really hard to test.
  370. ////
  371. The response includes a listing
  372. of any physical files recovered
  373. and their sizes.
  374. The response also includes timings in milliseconds
  375. of the various stages of recovery:
  376. * Index retrieval
  377. * Translog replay
  378. * Index start time
  379. This response indicates the recovery is `done`.
  380. All recoveries,
  381. whether ongoing or complete,
  382. are kept in the cluster state
  383. and may be reported on at any time.
  384. To only return information about ongoing recoveries,
  385. set the `active_only` query parameter to `true`.