repository-shared-file-system.asciidoc 4.4 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596
  1. [[snapshots-filesystem-repository]]
  2. === Shared file system repository
  3. include::{es-ref-dir}/snapshot-restore/on-prem-repo-type.asciidoc[]
  4. Use a shared file system repository to store snapshots on a shared file system.
  5. To register a shared file system repository, first mount the file system to the
  6. same location on all master and data nodes. Then add the file system's path or
  7. parent directory to the `path.repo` setting in `elasticsearch.yml` for each
  8. master and data node. For running clusters, this requires a
  9. <<restart-cluster-rolling,rolling restart>> of each node.
  10. Supported `path.repo` values vary by platform:
  11. include::{es-ref-dir}/tab-widgets/register-fs-repo-widget.asciidoc[]
  12. [[filesystem-repository-settings]]
  13. ==== Repository settings
  14. `chunk_size`::
  15. (Optional, <<byte-units,byte value>>)
  16. Maximum size of files in snapshots. In snapshots, files larger than this are
  17. broken down into chunks of this size or smaller. Defaults to `null` (unlimited
  18. file size).
  19. `compress`::
  20. (Optional, Boolean)
  21. If `true`, metadata files, such as index mappings and settings, are compressed
  22. in snapshots. Data files are not compressed. Defaults to `true`.
  23. `location`::
  24. (Required, string)
  25. Location of the shared filesystem used to store and retrieve snapshots. This
  26. location must be registered in the `path.repo` setting on all master and data
  27. nodes in the cluster.
  28. Unlike `path.repo`, this setting supports only a single file path.
  29. `max_number_of_snapshots`::
  30. (Optional, integer)
  31. Maximum number of snapshots the repository can contain.
  32. Defaults to `Integer.MAX_VALUE`, which is `2^31-1` or `2147483647`.
  33. include::repository-shared-settings.asciidoc[]
  34. ==== Troubleshooting a shared file system repository
  35. {es} interacts with a shared file system repository using the file system
  36. abstraction in your operating system. This means that every {es} node must be
  37. able to perform operations within the repository path such as creating,
  38. opening, and renaming files, and creating and listing directories, and
  39. operations performed by one node must be visible to other nodes as soon as they
  40. complete.
  41. Check for common misconfigurations using the <<verify-snapshot-repo-api>> API
  42. and the <<repo-analysis-api>> API. When the repository is properly configured,
  43. these APIs will complete successfully. If the verify repository or repository
  44. analysis APIs report a problem then you will be able to reproduce this problem
  45. outside {es} by performing similar operations on the file system directly.
  46. If the verify repository or repository analysis APIs fail with an error
  47. indicating insufficient permissions then adjust the configuration of the
  48. repository within your operating system to give {es} an appropriate level of
  49. access. To reproduce such problems directly, perform the same operations as
  50. {es} in the same security context as the one in which {es} is running. For
  51. example, on Linux, use a command such as `su` to switch to the user as which
  52. {es} runs.
  53. If the verify repository or repository analysis APIs fail with an error
  54. indicating that operations on one node are not immediately visible on another
  55. node then adjust the configuration of the repository within your operating
  56. system to address this problem. If your repository cannot be configured with
  57. strong enough visibility guarantees then it is not suitable for use as an {es}
  58. snapshot repository.
  59. The verify repository and repository analysis APIs will also fail if the
  60. operating system returns any other kind of I/O error when accessing the
  61. repository. If this happens, address the cause of the I/O error reported by the
  62. operating system.
  63. TIP: Many NFS implementations match accounts across nodes using their _numeric_
  64. user IDs (UIDs) and group IDs (GIDs) rather than their names. It is possible
  65. for {es} to run under an account with the same name (often `elasticsearch`) on
  66. each node, but for these accounts to have different numeric user or group IDs.
  67. If your shared file system uses NFS then ensure that every node is running with
  68. the same numeric UID and GID, or else update your NFS configuration to account
  69. for the variance in numeric IDs across nodes.
  70. [[repository-fs-linearizable-registers]]
  71. ==== Linearizable register implementation
  72. The linearizable register implementation for shared filesystem repositories is
  73. based around file locking. To perform a compare-and-exchange operation on a
  74. register, {es} first locks he underlying file and then writes the updated
  75. contents under the same lock. This ensures that the file has not changed in the
  76. meantime.