snapshots.asciidoc 32 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774
  1. [[modules-snapshots]]
  2. == Snapshot And Restore
  3. A snapshot is a backup taken from a running Elasticsearch cluster. You can take
  4. a snapshot of individual indices or of the entire cluster and store it in a
  5. repository on a shared filesystem, and there are plugins that support remote
  6. repositories on S3, HDFS, Azure, Google Cloud Storage and more.
  7. Snapshots are taken incrementally. This means that when creating a snapshot of
  8. an index Elasticsearch will avoid copying any data that is already stored in
  9. the repository as part of an earlier snapshot of the same index. Therefore it
  10. can be efficient to take snapshots of your cluster quite frequently.
  11. Snapshots can be restored into a running cluster via the restore API. When
  12. restoring an index it is possible to alter the name of the restored index as
  13. well as some of its settings, allowing a great deal of flexibility in how the
  14. snapshot and restore functionality can be used.
  15. WARNING: It is not possible to back up an Elasticsearch cluster simply by
  16. taking a copy of the data directories of all of its nodes. Elasticsearch may be
  17. making changes to the contents of its data directories while it is running, and
  18. this means that copying its data directories cannot be expected to capture a
  19. consistent picture of their contents. Attempts to restore a cluster from such a
  20. backup may fail, reporting corruption and/or missing files, or may appear to
  21. have succeeded having silently lost some of its data. The only reliable way to
  22. back up a cluster is by using the snapshot and restore functionality.
  23. [float]
  24. === Version compatibility
  25. IMPORTANT: Version compatibility refers to the underlying Lucene index
  26. compatibility. Follow the <<setup-upgrade,Upgrade documentation>>
  27. when migrating between versions.
  28. A snapshot contains a copy of the on-disk data structures that make up an
  29. index. This means that snapshots can only be restored to versions of
  30. Elasticsearch that can read the indices:
  31. * A snapshot of an index created in 5.x can be restored to 6.x.
  32. * A snapshot of an index created in 2.x can be restored to 5.x.
  33. * A snapshot of an index created in 1.x can be restored to 2.x.
  34. Conversely, snapshots of indices created in 1.x **cannot** be restored to 5.x
  35. or 6.x, and snapshots of indices created in 2.x **cannot** be restored to 6.x.
  36. Each snapshot can contain indices created in various versions of Elasticsearch,
  37. and when restoring a snapshot it must be possible to restore all of the indices
  38. into the target cluster. If any indices in a snapshot were created in an
  39. incompatible version, you will not be able restore the snapshot.
  40. IMPORTANT: When backing up your data prior to an upgrade, keep in mind that you
  41. won't be able to restore snapshots after you upgrade if they contain indices
  42. created in a version that's incompatible with the upgrade version.
  43. If you end up in a situation where you need to restore a snapshot of an index
  44. that is incompatible with the version of the cluster you are currently running,
  45. you can restore it on the latest compatible version and use
  46. <<reindex-from-remote,reindex-from-remote>> to rebuild the index on the current
  47. version. Reindexing from remote is only possible if the original index has
  48. source enabled. Retrieving and reindexing the data can take significantly
  49. longer than simply restoring a snapshot. If you have a large amount of data, we
  50. recommend testing the reindex from remote process with a subset of your data to
  51. understand the time requirements before proceeding.
  52. [float]
  53. === Repositories
  54. You must register a snapshot repository before you can perform snapshot and
  55. restore operations. We recommend creating a new snapshot repository for each
  56. major version. The valid repository settings depend on the repository type.
  57. If you register same snapshot repository with multiple clusters, only
  58. one cluster should have write access to the repository. All other clusters
  59. connected to that repository should set the repository to `readonly` mode.
  60. IMPORTANT: The snapshot format can change across major versions, so if you have
  61. clusters on different versions trying to write the same repository, snapshots
  62. written by one version may not be visible to the other and the repository could
  63. be corrupted. While setting the repository to `readonly` on all but one of the
  64. clusters should work with multiple clusters differing by one major version, it
  65. is not a supported configuration.
  66. [source,js]
  67. -----------------------------------
  68. PUT /_snapshot/my_backup
  69. {
  70. "type": "fs",
  71. "settings": {
  72. "location": "my_backup_location"
  73. }
  74. }
  75. -----------------------------------
  76. // CONSOLE
  77. // TESTSETUP
  78. To retrieve information about a registered repository, use a GET request:
  79. [source,js]
  80. -----------------------------------
  81. GET /_snapshot/my_backup
  82. -----------------------------------
  83. // CONSOLE
  84. which returns:
  85. [source,js]
  86. -----------------------------------
  87. {
  88. "my_backup": {
  89. "type": "fs",
  90. "settings": {
  91. "location": "my_backup_location"
  92. }
  93. }
  94. }
  95. -----------------------------------
  96. // TESTRESPONSE
  97. To retrieve information about multiple repositories, specify a comma-delimited
  98. list of repositories. You can also use the * wildcard when
  99. specifying repository names. For example, the following request retrieves
  100. information about all of the snapshot repositories that start with `repo` or
  101. contain `backup`:
  102. [source,js]
  103. -----------------------------------
  104. GET /_snapshot/repo*,*backup*
  105. -----------------------------------
  106. // CONSOLE
  107. To retrieve information about all registered snapshot repositories, omit the
  108. repository name or specify `_all`:
  109. [source,js]
  110. -----------------------------------
  111. GET /_snapshot
  112. -----------------------------------
  113. // CONSOLE
  114. or
  115. [source,js]
  116. -----------------------------------
  117. GET /_snapshot/_all
  118. -----------------------------------
  119. // CONSOLE
  120. [float]
  121. ===== Shared File System Repository
  122. The shared file system repository (`"type": "fs"`) uses the shared file system to store snapshots. In order to register
  123. the shared file system repository it is necessary to mount the same shared filesystem to the same location on all
  124. master and data nodes. This location (or one of its parent directories) must be registered in the `path.repo`
  125. setting on all master and data nodes.
  126. Assuming that the shared filesystem is mounted to `/mount/backups/my_fs_backup_location`, the following setting should
  127. be added to `elasticsearch.yml` file:
  128. [source,yaml]
  129. --------------
  130. path.repo: ["/mount/backups", "/mount/longterm_backups"]
  131. --------------
  132. The `path.repo` setting supports Microsoft Windows UNC paths as long as at least server name and share are specified as
  133. a prefix and back slashes are properly escaped:
  134. [source,yaml]
  135. --------------
  136. path.repo: ["\\\\MY_SERVER\\Snapshots"]
  137. --------------
  138. After all nodes are restarted, the following command can be used to register the shared file system repository with
  139. the name `my_fs_backup`:
  140. [source,js]
  141. -----------------------------------
  142. PUT /_snapshot/my_fs_backup
  143. {
  144. "type": "fs",
  145. "settings": {
  146. "location": "/mount/backups/my_fs_backup_location",
  147. "compress": true
  148. }
  149. }
  150. -----------------------------------
  151. // CONSOLE
  152. // TEST[skip:no access to absolute path]
  153. If the repository location is specified as a relative path this path will be resolved against the first path specified
  154. in `path.repo`:
  155. [source,js]
  156. -----------------------------------
  157. PUT /_snapshot/my_fs_backup
  158. {
  159. "type": "fs",
  160. "settings": {
  161. "location": "my_fs_backup_location",
  162. "compress": true
  163. }
  164. }
  165. -----------------------------------
  166. // CONSOLE
  167. // TEST[continued]
  168. The following settings are supported:
  169. [horizontal]
  170. `location`:: Location of the snapshots. Mandatory.
  171. `compress`:: Turns on compression of the snapshot files. Compression is applied only to metadata files (index mapping and settings). Data files are not compressed. Defaults to `true`.
  172. `chunk_size`:: Big files can be broken down into chunks during snapshotting if needed. The chunk size can be specified in bytes or by
  173. using size value notation, i.e. 1g, 10m, 5k. Defaults to `null` (unlimited chunk size).
  174. `max_restore_bytes_per_sec`:: Throttles per node restore rate. Defaults to `40mb` per second.
  175. `max_snapshot_bytes_per_sec`:: Throttles per node snapshot rate. Defaults to `40mb` per second.
  176. `readonly`:: Makes repository read-only. Defaults to `false`.
  177. [float]
  178. ===== Read-only URL Repository
  179. The URL repository (`"type": "url"`) can be used as an alternative read-only way to access data created by the shared file
  180. system repository. The URL specified in the `url` parameter should point to the root of the shared filesystem repository.
  181. The following settings are supported:
  182. [horizontal]
  183. `url`:: Location of the snapshots. Mandatory.
  184. URL Repository supports the following protocols: "http", "https", "ftp", "file" and "jar". URL repositories with `http:`,
  185. `https:`, and `ftp:` URLs has to be whitelisted by specifying allowed URLs in the `repositories.url.allowed_urls` setting.
  186. This setting supports wildcards in the place of host, path, query, and fragment. For example:
  187. [source,yaml]
  188. -----------------------------------
  189. repositories.url.allowed_urls: ["http://www.example.org/root/*", "https://*.mydomain.com/*?*#*"]
  190. -----------------------------------
  191. URL repositories with `file:` URLs can only point to locations registered in the `path.repo` setting similar to
  192. shared file system repository.
  193. [float]
  194. [role="xpack"]
  195. [testenv="basic"]
  196. ===== Source Only Repository
  197. A source repository enables you to create minimal, source-only snapshots that take up to 50% less space on disk.
  198. Source only snapshots contain stored fields and index metadata. They do not include index or doc values structures
  199. and are not searchable when restored. After restoring a source-only snapshot, you must <<docs-reindex,reindex>>
  200. the data into a new index.
  201. Source repositories delegate to another snapshot repository for storage.
  202. [IMPORTANT]
  203. ==================================================
  204. Source only snapshots are only supported if the `_source` field is enabled and no source-filtering is applied.
  205. When you restore a source only snapshot:
  206. * The restored index is read-only and can only serve `match_all` search or scroll requests to enable reindexing.
  207. * Queries other than `match_all` and `_get` requests are not supported.
  208. * The mapping of the restored index is empty, but the original mapping is available from the types top
  209. level `meta` element.
  210. ==================================================
  211. When you create a source repository, you must specify the type and name of the delegate repository
  212. where the snapshots will be stored:
  213. [source,js]
  214. -----------------------------------
  215. PUT _snapshot/my_src_only_repository
  216. {
  217. "type": "source",
  218. "settings": {
  219. "delegate_type": "fs",
  220. "location": "my_backup_location"
  221. }
  222. }
  223. -----------------------------------
  224. // CONSOLE
  225. // TEST[continued]
  226. [float]
  227. ===== Repository plugins
  228. Other repository backends are available in these official plugins:
  229. * {plugins}/repository-s3.html[repository-s3] for S3 repository support
  230. * {plugins}/repository-hdfs.html[repository-hdfs] for HDFS repository support in Hadoop environments
  231. * {plugins}/repository-azure.html[repository-azure] for Azure storage repositories
  232. * {plugins}/repository-gcs.html[repository-gcs] for Google Cloud Storage repositories
  233. [float]
  234. ===== Repository Verification
  235. When a repository is registered, it's immediately verified on all master and data nodes to make sure that it is functional
  236. on all nodes currently present in the cluster. The `verify` parameter can be used to explicitly disable the repository
  237. verification when registering or updating a repository:
  238. [source,js]
  239. -----------------------------------
  240. PUT /_snapshot/my_unverified_backup?verify=false
  241. {
  242. "type": "fs",
  243. "settings": {
  244. "location": "my_unverified_backup_location"
  245. }
  246. }
  247. -----------------------------------
  248. // CONSOLE
  249. // TEST[continued]
  250. The verification process can also be executed manually by running the following command:
  251. [source,js]
  252. -----------------------------------
  253. POST /_snapshot/my_unverified_backup/_verify
  254. -----------------------------------
  255. // CONSOLE
  256. // TEST[continued]
  257. It returns a list of nodes where repository was successfully verified or an error message if verification process failed.
  258. [float]
  259. === Snapshot
  260. A repository can contain multiple snapshots of the same cluster. Snapshots are identified by unique names within the
  261. cluster. A snapshot with the name `snapshot_1` in the repository `my_backup` can be created by executing the following
  262. command:
  263. [source,js]
  264. -----------------------------------
  265. PUT /_snapshot/my_backup/snapshot_1?wait_for_completion=true
  266. -----------------------------------
  267. // CONSOLE
  268. // TEST[continued]
  269. The `wait_for_completion` parameter specifies whether or not the request should return immediately after snapshot
  270. initialization (default) or wait for snapshot completion. During snapshot initialization, information about all
  271. previous snapshots is loaded into the memory, which means that in large repositories it may take several seconds (or
  272. even minutes) for this command to return even if the `wait_for_completion` parameter is set to `false`.
  273. By default a snapshot of all open and started indices in the cluster is created. This behavior can be changed by
  274. specifying the list of indices in the body of the snapshot request.
  275. [source,js]
  276. -----------------------------------
  277. PUT /_snapshot/my_backup/snapshot_2?wait_for_completion=true
  278. {
  279. "indices": "index_1,index_2",
  280. "ignore_unavailable": true,
  281. "include_global_state": false,
  282. "_meta": {
  283. "taken_by": "kimchy",
  284. "taken_because": "backup before upgrading"
  285. }
  286. }
  287. -----------------------------------
  288. // CONSOLE
  289. // TEST[continued]
  290. The list of indices that should be included into the snapshot can be specified using the `indices` parameter that
  291. supports <<multi-index,multi index syntax>>. The snapshot request also supports the
  292. `ignore_unavailable` option. Setting it to `true` will cause indices that do not exist to be ignored during snapshot
  293. creation. By default, when `ignore_unavailable` option is not set and an index is missing the snapshot request will fail.
  294. By setting `include_global_state` to false it's possible to prevent the cluster global state to be stored as part of
  295. the snapshot. By default, the entire snapshot will fail if one or more indices participating in the snapshot don't have
  296. all primary shards available. This behaviour can be changed by setting `partial` to `true`.
  297. The `_meta` field can be used to attach arbitrary metadata to the snapshot. This may be a record of who took the snapshot,
  298. why it was taken, or any other data that might be useful.
  299. Snapshot names can be automatically derived using <<date-math-index-names,date math expressions>>, similarly as when creating
  300. new indices. Note that special characters need to be URI encoded.
  301. For example, creating a snapshot with the current day in the name, like `snapshot-2018.05.11`, can be achieved with
  302. the following command:
  303. [source,js]
  304. -----------------------------------
  305. # PUT /_snapshot/my_backup/<snapshot-{now/d}>
  306. PUT /_snapshot/my_backup/%3Csnapshot-%7Bnow%2Fd%7D%3E
  307. -----------------------------------
  308. // CONSOLE
  309. // TEST[continued]
  310. The index snapshot process is incremental. In the process of making the index snapshot Elasticsearch analyses
  311. the list of the index files that are already stored in the repository and copies only files that were created or
  312. changed since the last snapshot. That allows multiple snapshots to be preserved in the repository in a compact form.
  313. Snapshotting process is executed in non-blocking fashion. All indexing and searching operation can continue to be
  314. executed against the index that is being snapshotted. However, a snapshot represents the point-in-time view of the index
  315. at the moment when snapshot was created, so no records that were added to the index after the snapshot process was started
  316. will be present in the snapshot. The snapshot process starts immediately for the primary shards that has been started
  317. and are not relocating at the moment. Before version 1.2.0, the snapshot operation fails if the cluster has any relocating or
  318. initializing primaries of indices participating in the snapshot. Starting with version 1.2.0, Elasticsearch waits for
  319. relocation or initialization of shards to complete before snapshotting them.
  320. Besides creating a copy of each index the snapshot process can also store global cluster metadata, which includes persistent
  321. cluster settings and templates. The transient settings and registered snapshot repositories are not stored as part of
  322. the snapshot.
  323. Only one snapshot process can be executed in the cluster at any time. While snapshot of a particular shard is being
  324. created this shard cannot be moved to another node, which can interfere with rebalancing process and allocation
  325. filtering. Elasticsearch will only be able to move a shard to another node (according to the current allocation
  326. filtering settings and rebalancing algorithm) once the snapshot is finished.
  327. Once a snapshot is created information about this snapshot can be obtained using the following command:
  328. [source,sh]
  329. -----------------------------------
  330. GET /_snapshot/my_backup/snapshot_1
  331. -----------------------------------
  332. // CONSOLE
  333. // TEST[continued]
  334. This command returns basic information about the snapshot including start and end time, version of
  335. Elasticsearch that created the snapshot, the list of included indices, the current state of the
  336. snapshot and the list of failures that occurred during the snapshot. The snapshot `state` can be
  337. [horizontal]
  338. `IN_PROGRESS`::
  339. The snapshot is currently running.
  340. `SUCCESS`::
  341. The snapshot finished and all shards were stored successfully.
  342. `FAILED`::
  343. The snapshot finished with an error and failed to store any data.
  344. `PARTIAL`::
  345. The global cluster state was stored, but data of at least one shard wasn't stored successfully.
  346. The `failure` section in this case should contain more detailed information about shards
  347. that were not processed correctly.
  348. `INCOMPATIBLE`::
  349. The snapshot was created with an old version of Elasticsearch and therefore is incompatible with
  350. the current version of the cluster.
  351. Similar as for repositories, information about multiple snapshots can be queried in one go, supporting wildcards as well:
  352. [source,sh]
  353. -----------------------------------
  354. GET /_snapshot/my_backup/snapshot_*,some_other_snapshot
  355. -----------------------------------
  356. // CONSOLE
  357. // TEST[continued]
  358. All snapshots currently stored in the repository can be listed using the following command:
  359. [source,sh]
  360. -----------------------------------
  361. GET /_snapshot/my_backup/_all
  362. -----------------------------------
  363. // CONSOLE
  364. // TEST[continued]
  365. The command fails if some of the snapshots are unavailable. The boolean parameter `ignore_unavailable` can be used to
  366. return all snapshots that are currently available.
  367. Getting all snapshots in the repository can be costly on cloud-based repositories,
  368. both from a cost and performance perspective. If the only information required is
  369. the snapshot names/uuids in the repository and the indices in each snapshot, then
  370. the optional boolean parameter `verbose` can be set to `false` to execute a more
  371. performant and cost-effective retrieval of the snapshots in the repository. Note
  372. that setting `verbose` to `false` will omit all other information about the snapshot
  373. such as status information, the number of snapshotted shards, etc. The default
  374. value of the `verbose` parameter is `true`.
  375. A currently running snapshot can be retrieved using the following command:
  376. [source,sh]
  377. -----------------------------------
  378. GET /_snapshot/my_backup/_current
  379. -----------------------------------
  380. // CONSOLE
  381. // TEST[continued]
  382. A snapshot can be deleted from the repository using the following command:
  383. [source,sh]
  384. -----------------------------------
  385. DELETE /_snapshot/my_backup/snapshot_2
  386. -----------------------------------
  387. // CONSOLE
  388. // TEST[continued]
  389. When a snapshot is deleted from a repository, Elasticsearch deletes all files that are associated with the deleted
  390. snapshot and not used by any other snapshots. If the deleted snapshot operation is executed while the snapshot is being
  391. created the snapshotting process will be aborted and all files created as part of the snapshotting process will be
  392. cleaned. Therefore, the delete snapshot operation can be used to cancel long running snapshot operations that were
  393. started by mistake.
  394. A repository can be unregistered using the following command:
  395. [source,sh]
  396. -----------------------------------
  397. DELETE /_snapshot/my_backup
  398. -----------------------------------
  399. // CONSOLE
  400. // TEST[continued]
  401. When a repository is unregistered, Elasticsearch only removes the reference to the location where the repository is storing
  402. the snapshots. The snapshots themselves are left untouched and in place.
  403. [float]
  404. === Restore
  405. A snapshot can be restored using the following command:
  406. [source,sh]
  407. -----------------------------------
  408. POST /_snapshot/my_backup/snapshot_1/_restore
  409. -----------------------------------
  410. // CONSOLE
  411. // TEST[continued]
  412. By default, all indices in the snapshot are restored, and the cluster state is
  413. *not* restored. It's possible to select indices that should be restored as well
  414. as to allow the global cluster state from being restored by using `indices` and
  415. `include_global_state` options in the restore request body. The list of indices
  416. supports <<multi-index,multi index syntax>>. The `rename_pattern`
  417. and `rename_replacement` options can be also used to rename indices on restore
  418. using regular expression that supports referencing the original text as
  419. explained
  420. http://docs.oracle.com/javase/6/docs/api/java/util/regex/Matcher.html#appendReplacement(java.lang.StringBuffer,%20java.lang.String)[here].
  421. Set `include_aliases` to `false` to prevent aliases from being restored together
  422. with associated indices
  423. [source,js]
  424. -----------------------------------
  425. POST /_snapshot/my_backup/snapshot_1/_restore
  426. {
  427. "indices": "index_1,index_2",
  428. "ignore_unavailable": true,
  429. "include_global_state": true,
  430. "rename_pattern": "index_(.+)",
  431. "rename_replacement": "restored_index_$1"
  432. }
  433. -----------------------------------
  434. // CONSOLE
  435. // TEST[continued]
  436. The restore operation can be performed on a functioning cluster. However, an
  437. existing index can be only restored if it's <<indices-open-close,closed>> and
  438. has the same number of shards as the index in the snapshot. The restore
  439. operation automatically opens restored indices if they were closed and creates
  440. new indices if they didn't exist in the cluster. If cluster state is restored
  441. with `include_global_state` (defaults to `false`), the restored templates that
  442. don't currently exist in the cluster are added and existing templates with the
  443. same name are replaced by the restored templates. The restored persistent
  444. settings are added to the existing persistent settings.
  445. [float]
  446. ==== Partial restore
  447. By default, the entire restore operation will fail if one or more indices participating in the operation don't have
  448. snapshots of all shards available. It can occur if some shards failed to snapshot for example. It is still possible to
  449. restore such indices by setting `partial` to `true`. Please note, that only successfully snapshotted shards will be
  450. restored in this case and all missing shards will be recreated empty.
  451. [float]
  452. ==== Changing index settings during restore
  453. Most of index settings can be overridden during the restore process. For example, the following command will restore
  454. the index `index_1` without creating any replicas while switching back to default refresh interval:
  455. [source,js]
  456. -----------------------------------
  457. POST /_snapshot/my_backup/snapshot_1/_restore
  458. {
  459. "indices": "index_1",
  460. "index_settings": {
  461. "index.number_of_replicas": 0
  462. },
  463. "ignore_index_settings": [
  464. "index.refresh_interval"
  465. ]
  466. }
  467. -----------------------------------
  468. // CONSOLE
  469. // TEST[continued]
  470. Please note, that some settings such as `index.number_of_shards` cannot be changed during restore operation.
  471. [float]
  472. ==== Restoring to a different cluster
  473. The information stored in a snapshot is not tied to a particular cluster or a cluster name. Therefore it's possible to
  474. restore a snapshot made from one cluster into another cluster. All that is required is registering the repository
  475. containing the snapshot in the new cluster and starting the restore process. The new cluster doesn't have to have the
  476. same size or topology. However, the version of the new cluster should be the same or newer (only 1 major version newer) than the cluster that was used to create the snapshot. For example, you can restore a 1.x snapshot to a 2.x cluster, but not a 1.x snapshot to a 5.x cluster.
  477. If the new cluster has a smaller size additional considerations should be made. First of all it's necessary to make sure
  478. that new cluster have enough capacity to store all indices in the snapshot. It's possible to change indices settings
  479. during restore to reduce the number of replicas, which can help with restoring snapshots into smaller cluster. It's also
  480. possible to select only subset of the indices using the `indices` parameter.
  481. If indices in the original cluster were assigned to particular nodes using
  482. <<shard-allocation-filtering,shard allocation filtering>>, the same rules will be enforced in the new cluster. Therefore
  483. if the new cluster doesn't contain nodes with appropriate attributes that a restored index can be allocated on, such
  484. index will not be successfully restored unless these index allocation settings are changed during restore operation.
  485. The restore operation also checks that restored persistent settings are compatible with the current cluster to avoid accidentally
  486. restoring incompatible settings. If you need to restore a snapshot with incompatible persistent settings, try restoring it without
  487. the global cluster state.
  488. [float]
  489. === Snapshot status
  490. A list of currently running snapshots with their detailed status information can be obtained using the following command:
  491. [source,sh]
  492. -----------------------------------
  493. GET /_snapshot/_status
  494. -----------------------------------
  495. // CONSOLE
  496. // TEST[continued]
  497. In this format, the command will return information about all currently running snapshots. By specifying a repository name, it's possible
  498. to limit the results to a particular repository:
  499. [source,sh]
  500. -----------------------------------
  501. GET /_snapshot/my_backup/_status
  502. -----------------------------------
  503. // CONSOLE
  504. // TEST[continued]
  505. If both repository name and snapshot id are specified, this command will return detailed status information for the given snapshot even
  506. if it's not currently running:
  507. [source,sh]
  508. -----------------------------------
  509. GET /_snapshot/my_backup/snapshot_1/_status
  510. -----------------------------------
  511. // CONSOLE
  512. // TEST[continued]
  513. The output looks similar to the following:
  514. [source,js]
  515. --------------------------------------------------
  516. {
  517. "snapshots": [
  518. {
  519. "snapshot": "snapshot_1",
  520. "repository": "my_backup",
  521. "uuid": "XuBo4l4ISYiVg0nYUen9zg",
  522. "state": "SUCCESS",
  523. "include_global_state": true,
  524. "shards_stats": {
  525. "initializing": 0,
  526. "started": 0,
  527. "finalizing": 0,
  528. "done": 5,
  529. "failed": 0,
  530. "total": 5
  531. },
  532. "stats": {
  533. "incremental": {
  534. "file_count": 8,
  535. "size_in_bytes": 4704
  536. },
  537. "processed": {
  538. "file_count": 7,
  539. "size_in_bytes": 4254
  540. },
  541. "total": {
  542. "file_count": 8,
  543. "size_in_bytes": 4704
  544. },
  545. "start_time_in_millis": 1526280280355,
  546. "time_in_millis": 358
  547. }
  548. }
  549. ]
  550. }
  551. --------------------------------------------------
  552. // TESTRESPONSE
  553. The output is composed of different sections. The `stats` sub-object provides details on the number and size of files that were
  554. snapshotted. As snapshots are incremental, copying only the Lucene segments that are not already in the repository,
  555. the `stats` object contains a `total` section for all the files that are referenced by the snapshot, as well as an `incremental` section
  556. for those files that actually needed to be copied over as part of the incremental snapshotting. In case of a snapshot that's still
  557. in progress, there's also a `processed` section that contains information about the files that are in the process of being copied.
  558. Multiple ids are also supported:
  559. [source,sh]
  560. -----------------------------------
  561. GET /_snapshot/my_backup/snapshot_1,snapshot_2/_status
  562. -----------------------------------
  563. // CONSOLE
  564. // TEST[continued]
  565. [float]
  566. [[monitor-snapshot-restore-progress]]
  567. === Monitoring snapshot/restore progress
  568. There are several ways to monitor the progress of the snapshot and restores processes while they are running. Both
  569. operations support `wait_for_completion` parameter that would block client until the operation is completed. This is
  570. the simplest method that can be used to get notified about operation completion.
  571. The snapshot operation can be also monitored by periodic calls to the snapshot info:
  572. [source,sh]
  573. -----------------------------------
  574. GET /_snapshot/my_backup/snapshot_1
  575. -----------------------------------
  576. // CONSOLE
  577. // TEST[continued]
  578. Please note that snapshot info operation uses the same resources and thread pool as the snapshot operation. So,
  579. executing a snapshot info operation while large shards are being snapshotted can cause the snapshot info operation to wait
  580. for available resources before returning the result. On very large shards the wait time can be significant.
  581. To get more immediate and complete information about snapshots the snapshot status command can be used instead:
  582. [source,sh]
  583. -----------------------------------
  584. GET /_snapshot/my_backup/snapshot_1/_status
  585. -----------------------------------
  586. // CONSOLE
  587. // TEST[continued]
  588. While snapshot info method returns only basic information about the snapshot in progress, the snapshot status returns
  589. complete breakdown of the current state for each shard participating in the snapshot.
  590. The restore process piggybacks on the standard recovery mechanism of the Elasticsearch. As a result, standard recovery
  591. monitoring services can be used to monitor the state of restore. When restore operation is executed the cluster
  592. typically goes into `red` state. It happens because the restore operation starts with "recovering" primary shards of the
  593. restored indices. During this operation the primary shards become unavailable which manifests itself in the `red` cluster
  594. state. Once recovery of primary shards is completed Elasticsearch is switching to standard replication process that
  595. creates the required number of replicas at this moment cluster switches to the `yellow` state. Once all required replicas
  596. are created, the cluster switches to the `green` states.
  597. The cluster health operation provides only a high level status of the restore process. It's possible to get more
  598. detailed insight into the current state of the recovery process by using <<indices-recovery, indices recovery>> and
  599. <<cat-recovery, cat recovery>> APIs.
  600. [float]
  601. === Stopping currently running snapshot and restore operations
  602. The snapshot and restore framework allows running only one snapshot or one restore operation at a time. If a currently
  603. running snapshot was executed by mistake, or takes unusually long, it can be terminated using the snapshot delete operation.
  604. The snapshot delete operation checks if the deleted snapshot is currently running and if it does, the delete operation stops
  605. that snapshot before deleting the snapshot data from the repository.
  606. [source,sh]
  607. -----------------------------------
  608. DELETE /_snapshot/my_backup/snapshot_1
  609. -----------------------------------
  610. // CONSOLE
  611. // TEST[continued]
  612. The restore operation uses the standard shard recovery mechanism. Therefore, any currently running restore operation can
  613. be canceled by deleting indices that are being restored. Please note that data for all deleted indices will be removed
  614. from the cluster as a result of this operation.
  615. [float]
  616. === Effect of cluster blocks on snapshot and restore operations
  617. Many snapshot and restore operations are affected by cluster and index blocks. For example, registering and unregistering
  618. repositories require write global metadata access. The snapshot operation requires that all indices and their metadata as
  619. well as the global metadata were readable. The restore operation requires the global metadata to be writable, however
  620. the index level blocks are ignored during restore because indices are essentially recreated during restore.
  621. Please note that a repository content is not part of the cluster and therefore cluster blocks don't affect internal
  622. repository operations such as listing or deleting snapshots from an already registered repository.