Browse Source

[DOCS] Improve docs for Windows DOS/UNC paths in `path.*` settings (#64668)

James Rodewig 5 years ago
parent
commit
357ab0073e

+ 29 - 29
docs/reference/setup/important-settings/path-settings.asciidoc

@@ -2,32 +2,32 @@
 [discrete]
 === Path settings
 
-If you are using the `.zip` or `.tar.gz` archives, the `data` and `logs`
-directories are sub-folders of `$ES_HOME`. If these important folders are left
-in their default locations, there is a high risk of them being deleted while
-upgrading {es} to a new version.
-
-In production use, you will almost certainly want to change the locations of the
-`path.data` and `path.logs` folders:
-
-[source,yaml]
---------------------------------------------------
-path:
-  logs: /var/log/elasticsearch
-  data: /var/data/elasticsearch
---------------------------------------------------
-
-The RPM and Debian distributions already use custom paths for `data` and `logs`.
-
-The `path.data` settings can be set to multiple paths, in which case all paths
-will be used to store data. However, the files belonging to a single shard will
-all be stored on the same data path:
-
-[source,yaml]
---------------------------------------------------
-path:
-  data:
-    - /mnt/elasticsearch_1
-    - /mnt/elasticsearch_2
-    - /mnt/elasticsearch_3
---------------------------------------------------
+For <<targz,macOS `.tar.gz`>>, <<targz,Linux `.tar.gz`>>, and
+<<zip-windows,Windows `.zip`>> installations, {es} writes data and logs to the
+respective `data` and `logs` subdirectories of `$ES_HOME` by default.
+However, files in `$ES_HOME` risk deletion during an upgrade.
+
+In production, we strongly recommend you set the `path.data` and `path.logs` in
+`elasticsearch.yml` to locations outside of `$ES_HOME`.
+
+TIP: <<docker,Docker>>, <<deb,Debian>>, <<rpm,RPM>>, <<brew,macOS Homebrew>>,
+and <<windows,Windows `.msi`>> installations write data and log to locations
+outside of `$ES_HOME` by default.
+
+Supported `path.data` and `path.logs` values vary by platform:
+
+include::{es-repo-dir}/tab-widgets/code.asciidoc[]
+
+include::{es-repo-dir}/tab-widgets/customize-data-log-path-widget.asciidoc[]
+
+If needed, you can specify multiple paths in `path.data`. {es} stores the node's
+data across all provided paths but keeps each shard's data on the same path.
+
+WARNING: {es} does not balance shards across a node's data paths. High disk
+usage in a single path can trigger a <<disk-based-shard-allocation,high disk
+usage watermark>> for the entire node. If triggered, {es} will not add shards to
+the node, even if the node’s other paths have available disk space. If you need
+additional disk space, we recommend you add a new node rather than additional
+data paths. 
+
+include::{es-repo-dir}/tab-widgets/multi-data-path-widget.asciidoc[]

+ 10 - 61
docs/reference/snapshot-restore/register-repository.asciidoc

@@ -100,71 +100,20 @@ are left untouched and in place.
 [[snapshots-filesystem-repository]]
 === Shared file system repository
 
-The shared file system repository (`"type": "fs"`) uses the shared file system to store snapshots. In order to register
-the shared file system repository it is necessary to mount the same shared filesystem to the same location on all
-master and data nodes. This location (or one of its parent directories) must be registered in the `path.repo`
-setting on all master and data nodes.
+Use a shared file system repository (`"type": "fs"`) to store snapshots on a
+shared file system.
 
-Assuming that the shared filesystem is mounted to `/mount/backups/my_fs_backup_location`, the following setting should
-be added to `elasticsearch.yml` file:
+To register a shared file system repository, first mount the file system to the
+same location on all master and data nodes. Then add the file system's
+path or parent directory to the `path.repo` setting in `elasticsearch.yml` for
+each master and data node. For running clusters, this requires a
+<<restart-cluster-rolling,rolling restart>> of each node.
 
-[source,yaml]
---------------
-path.repo: ["/mount/backups", "/mount/longterm_backups"]
---------------
-
-The `path.repo` setting supports Microsoft Windows UNC paths as long as at least server name and share are specified as
-a prefix and back slashes are properly escaped:
-
-[source,yaml]
---------------
-path.repo: ["\\\\MY_SERVER\\Snapshots"]
---------------
-
-After all nodes are restarted, the following command can be used to register the shared file system repository with
-the name `my_fs_backup`:
-
-[source,console]
------------------------------------
-PUT /_snapshot/my_fs_backup
-{
-  "type": "fs",
-  "settings": {
-    "location": "/mount/backups/my_fs_backup_location",
-    "compress": true
-  }
-}
------------------------------------
-// TEST[skip:no access to absolute path]
-
-If the repository location is specified as a relative path this path will be resolved against the first path specified
-in `path.repo`:
+Supported `path.repo` values vary by platform:
 
-[source,console]
------------------------------------
-PUT /_snapshot/my_fs_backup
-{
-  "type": "fs",
-  "settings": {
-    "location": "my_fs_backup_location",
-    "compress": true
-  }
-}
------------------------------------
-// TEST[continued]
+include::{es-repo-dir}/tab-widgets/code.asciidoc[]
 
-The following settings are supported:
-
-`location`:: Location of the snapshots. Mandatory.
-`compress`:: Turns on compression of the snapshot files. Compression is applied only to metadata files (index mapping and settings). Data files are not compressed. Defaults to `true`.
-`chunk_size`:: Big files can be broken down into chunks during snapshotting if needed. Specify the chunk size as a value and
-unit, for example: `1GB`, `10MB`, `5KB`, `500B`. Defaults to `null` (unlimited chunk size).
-`max_restore_bytes_per_sec`:: Throttles per node restore rate. Defaults to unlimited. Note that restores are also throttled through <<recovery,recovery settings>>.
-`max_snapshot_bytes_per_sec`:: Throttles per node snapshot rate. Defaults to `40mb` per second.
-`readonly`:: Makes repository read-only.  Defaults to `false`.
-`max_number_of_snapshots`:: Limits the maximum number of snapshots that the repository may contain. Defaults to `500`. Note that snapshot repositories do not
-scale indefinitely in size and might lead to master node performance and stability issues if they grow past a certain size. We do not recommend increasing this setting.
-Instead you should delete older snapshots or use multiple repositories.
+include::{es-repo-dir}/tab-widgets/register-fs-repo-widget.asciidoc[]
 
 [discrete]
 [[snapshots-read-only-repository]]

+ 39 - 0
docs/reference/tab-widgets/customize-data-log-path-widget.asciidoc

@@ -0,0 +1,39 @@
+++++
+<div class="tabs" data-tab-group="os">
+  <div role="tablist" aria-label="data-log-path">
+    <button role="tab"
+            aria-selected="true"
+            aria-controls="unix-tab-data-log-path"
+            id="unix-data-log-path">
+      Unix-like systems
+    </button>
+    <button role="tab"
+            aria-selected="false"
+            aria-controls="win-tab-data-log-path"
+            id="win-data-log-path">
+      Windows
+    </button>
+  </div>
+  <div tabindex="0"
+       role="tabpanel"
+       id="unix-tab-data-log-path"
+       aria-labelledby="unix-data-log-path">
+++++
+
+include::customize-data-log-path.asciidoc[tag=unix]
+
+++++
+  </div>
+  <div tabindex="0"
+       role="tabpanel"
+       id="win-tab-data-log-path"
+       aria-labelledby="win-data-log-path"
+       hidden="">
+++++
+
+include::customize-data-log-path.asciidoc[tag=win]
+
+++++
+  </div>
+</div>
+++++

+ 22 - 0
docs/reference/tab-widgets/customize-data-log-path.asciidoc

@@ -0,0 +1,22 @@
+// tag::unix[]
+Linux and macOS installations support Unix-style paths:
+
+[source,yaml]
+----
+path:
+  data: /var/data/elasticsearch
+  logs: /var/log/elasticsearch
+----
+// end::unix[]
+
+
+// tag::win[]
+Windows installations support DOS paths with escaped backslashes:
+
+[source,yaml]
+----
+path:
+  data: "C:\\Elastic\\Elasticsearch\\data"
+  logs: "C:\\Elastic\\Elasticsearch\\logs"
+----
+// end::win[]

+ 1 - 1
docs/reference/tab-widgets/logging-widget.asciidoc

@@ -25,7 +25,7 @@
             aria-controls="mac-tab-logs"
             id="mac-logs"
             tabindex="-1">
-      MacOS
+      macOS
     </button>
     <button role="tab"
             aria-selected="false"

+ 15 - 3
docs/reference/tab-widgets/logging.asciidoc

@@ -12,26 +12,38 @@ For <<rpm,RPM installations>>, {es} writes logs to `/var/log/elasticsearch`.
 // end::rpm[]
 
 // tag::mac[]
-For <<targz,MacOS `.tar.gz`>> installations, {es} writes logs to
+For <<targz,macOS `.tar.gz`>> installations, {es} writes logs to
 `$ES_HOME/logs`.
+
+Files in `$ES_HOME` risk deletion during an upgrade. In production, we strongly
+recommend you set `path.logs` to a location outside of `$ES_HOME`.
+See <<path-settings>>.
 // end::mac[]
 
 // tag::brew[]
-For <<brew,MacOS Homebrew>> installations, {es} writes logs to
+For <<brew,macOS Homebrew>> installations, {es} writes logs to
 `/usr/local/var/log/elasticsearch`.
 // end::brew[]
 
 // tag::linux[]
 For <<targz,Linux `.tar.gz`>> installations, {es} writes logs to
 `$ES_HOME/logs`.
+
+Files in `$ES_HOME` risk deletion during an upgrade. In production, we strongly
+recommend you set `path.logs` to a location outside of `$ES_HOME`.
+See <<path-settings>>.
 // end::linux[]
 
 // tag::win-zip[]
 For <<zip-windows,Windows `.zip`>> installations, {es} writes logs to
 `%ES_HOME%\logs`.
+
+Files in `%ES_HOME%` risk deletion during an upgrade. In production, we strongly
+recommend you set `path.logs` to a location outside of `%ES_HOME%``.
+See <<path-settings>>.
 // end::win-zip[]
 
 // tag::win-msi[]
 For <<windows,Windows `.msi`>> installations, {es} writes logs to
 `%ALLUSERSPROFILE%\Elastic\Elasticsearch\logs`.
-// end::win-msi[]
+// end::win-msi[]

+ 39 - 0
docs/reference/tab-widgets/multi-data-path-widget.asciidoc

@@ -0,0 +1,39 @@
+++++
+<div class="tabs" data-tab-group="os">
+  <div role="tablist" aria-label="multi-data-path">
+    <button role="tab"
+            aria-selected="true"
+            aria-controls="unix-tab-multi-data-path"
+            id="unix-multi-data-path">
+      Unix-like systems
+    </button>
+    <button role="tab"
+            aria-selected="false"
+            aria-controls="win-tab-multi-data-path"
+            id="win-multi-data-path">
+      Windows
+    </button>
+  </div>
+  <div tabindex="0"
+       role="tabpanel"
+       id="unix-tab-multi-data-path"
+       aria-labelledby="unix-multi-data-path">
+++++
+
+include::multi-data-path.asciidoc[tag=unix]
+
+++++
+  </div>
+  <div tabindex="0"
+       role="tabpanel"
+       id="win-tab-multi-data-path"
+       aria-labelledby="win-multi-data-path"
+       hidden="">
+++++
+
+include::multi-data-path.asciidoc[tag=win]
+
+++++
+  </div>
+</div>
+++++

+ 26 - 0
docs/reference/tab-widgets/multi-data-path.asciidoc

@@ -0,0 +1,26 @@
+// tag::unix[]
+Linux and macOS installations support multiple Unix-style paths in `path.data`:
+
+[source,yaml]
+----
+path:
+  data:
+    - /mnt/elasticsearch_1
+    - /mnt/elasticsearch_2
+    - /mnt/elasticsearch_3
+----
+// end::unix[]
+
+
+// tag::win[]
+Windows installations support multiple DOS paths in `path.data`:
+
+[source,yaml]
+----
+path:
+  data:
+    - "C:\\Elastic\\Elasticsearch_1"
+    - "E:\\Elastic\\Elasticsearch_1"
+    - "F:\\Elastic\\Elasticsearch_3"
+----
+// end::win[]

+ 39 - 0
docs/reference/tab-widgets/register-fs-repo-widget.asciidoc

@@ -0,0 +1,39 @@
+++++
+<div class="tabs" data-tab-group="os">
+  <div role="tablist" aria-label="fs-repo">
+    <button role="tab"
+            aria-selected="true"
+            aria-controls="unix-tab-fs-repo"
+            id="unix-fs-repo">
+      Unix-like systems
+    </button>
+    <button role="tab"
+            aria-selected="false"
+            aria-controls="win-tab-fs-repo"
+            id="win-fs-repo">
+      Windows
+    </button>
+  </div>
+  <div tabindex="0"
+       role="tabpanel"
+       id="unix-tab-fs-repo"
+       aria-labelledby="unix-fs-repo">
+++++
+
+include::register-fs-repo.asciidoc[tag=unix]
+
+++++
+  </div>
+  <div tabindex="0"
+       role="tabpanel"
+       id="win-tab-fs-repo"
+       aria-labelledby="win-fs-repo"
+       hidden="">
+++++
+
+include::register-fs-repo.asciidoc[tag=win]
+
+++++
+  </div>
+</div>
+++++

+ 103 - 0
docs/reference/tab-widgets/register-fs-repo.asciidoc

@@ -0,0 +1,103 @@
+// tag::unix[]
+Linux and macOS installations support Unix-style paths:
+
+[source,yaml]
+----
+path:
+  repo:
+    - /mount/backups
+    - /mount/long_term_backups
+----
+
+After restarting each node, use the <<put-snapshot-repo-api,put snapshot
+repository>> API to register the file system repository. Specify the file
+system's path in `settings.location`:
+
+[source,console]
+----
+PUT /_snapshot/my_fs_backup
+{
+  "type": "fs",
+  "settings": {
+    "location": "/mount/backups/my_fs_backup_location",
+    "compress": true
+  }
+}
+----
+// TEST[skip:no access to path]
+
+If you specify a relative path in `settings.location`, {es} resolves the path
+using the first value in the `path.repo` setting.
+
+[source,console]
+----
+PUT /_snapshot/my_fs_backup
+{
+  "type": "fs",
+  "settings": {
+    "location": "my_fs_backup_location",       <1>
+    "compress": true
+  }
+}
+----
+// TEST[skip:no access to path]
+
+<1> The first value in the `path.repo` setting is `/mount/backups`. This
+relative path, `my_fs_backup_location`, resolves to
+`/mount/backups/my_fs_backup_location`.
+// end::unix[]
+
+
+// tag::win[]
+Windows installations support both DOS and Microsoft UNC paths. Escaped any
+backslashes in the paths. For UNC paths, provide the server and share name as a
+prefix.
+
+[source,yaml]
+----
+path:
+  repo:
+    - "E:\\Mount\\Backups"                      <1>
+    - "\\\\MY_SERVER\\Mount\\Long_term_backups" <2>
+----
+
+<1> DOS path
+<2> UNC path
+
+After restarting each node, use the <<put-snapshot-repo-api,put snapshot
+repository>> API to register the file system repository. Specify the file
+system's path in `settings.location`:
+
+[source,console]
+----
+PUT /_snapshot/my_fs_backup
+{
+  "type": "fs",
+  "settings": {
+    "location": "E:\\Mount\\Backups\\My_fs_backup_location",
+    "compress": true
+  }
+}
+----
+// TEST[skip:no access to path]
+
+If you specify a relative path in `settings.location`, {es} resolves the path
+using the first value in the `path.repo` setting.
+
+[source,console]
+----
+PUT /_snapshot/my_fs_backup
+{
+  "type": "fs",
+  "settings": {
+    "location": "My_fs_backup_location",       <1>
+    "compress": true
+  }
+}
+----
+// TEST[skip:no access to path]
+
+<1> The first value in the `path.repo` setting is `E:\Mount\Backups`. This
+relative path, `My_fs_backup_location`, resolves to
+`E:\Mount\Backups\My_fs_backup_location`.
+// end::win[]