|
@@ -17,6 +17,8 @@ image::monitoring/images/metricbeat.png[Example monitoring architecture]
|
|
|
To learn about monitoring in general, see
|
|
|
{stack-ov}/xpack-monitoring.html[Monitoring the {stack}].
|
|
|
|
|
|
+//NOTE: The tagged regions are re-used in the Stack Overview.
|
|
|
+
|
|
|
. Enable the collection of monitoring data. Set
|
|
|
`xpack.monitoring.collection.enabled` to `true` on each node in the production
|
|
|
cluster. By default, it is is disabled (`false`).
|
|
@@ -71,13 +73,13 @@ PUT _cluster/settings
|
|
|
Leave `xpack.monitoring.enabled` set to its default value (`true`).
|
|
|
--
|
|
|
|
|
|
-. On each {es} node in the production cluster:
|
|
|
-
|
|
|
-.. {metricbeat-ref}/metricbeat-installation.html[Install {metricbeat}].
|
|
|
+. {metricbeat-ref}/metricbeat-installation.html[Install {metricbeat}] on each
|
|
|
+{es} node in the production cluster.
|
|
|
|
|
|
-.. Enable the {es} module in {metricbeat}. +
|
|
|
+. Enable the {es} {xpack} module in {metricbeat} on each {es} node. +
|
|
|
+
|
|
|
--
|
|
|
+// tag::enable-es-module[]
|
|
|
For example, to enable the default configuration in the `modules.d` directory,
|
|
|
run the following command:
|
|
|
|
|
@@ -89,39 +91,57 @@ metricbeat modules enable elasticsearch-xpack
|
|
|
For more information, see
|
|
|
{metricbeat-ref}/configuration-metricbeat.html[Specify which modules to run] and
|
|
|
{metricbeat-ref}/metricbeat-module-elasticsearch.html[{es} module].
|
|
|
---
|
|
|
-
|
|
|
-.. By default the module will collect {es} monitoring metrics from `http://localhost:9200`.
|
|
|
-If the local {es} node has a different address, you must specify it via the `hosts` setting
|
|
|
-in the `modules.d/elasticsearch-xpack.yml` file.
|
|
|
|
|
|
-.. If Elastic {security-features} are enabled, you must also provide a user ID
|
|
|
-and password so that {metricbeat} can collect metrics successfully.
|
|
|
-
|
|
|
-... Create a user on the production cluster that has the
|
|
|
-{stack-ov}/built-in-roles.html[`remote_monitoring_collector` built-in role].
|
|
|
-Alternatively, use the {stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user].
|
|
|
+// end::enable-es-module[]
|
|
|
+--
|
|
|
|
|
|
-... Add the `username` and `password` settings to the {es} module configuration
|
|
|
-file.
|
|
|
+. Configure the {es} {xpack} module in {metricbeat} on each {es} node. +
|
|
|
+
|
|
|
--
|
|
|
-For example, add the following settings in the `modules.d/elasticsearch-xpack.yml` file:
|
|
|
+// tag::configure-es-module[]
|
|
|
+The `modules.d/elasticsearch-xpack.yml` file contains the following settings:
|
|
|
|
|
|
[source,yaml]
|
|
|
----------------------------------
|
|
|
-- module: elasticsearch
|
|
|
- ...
|
|
|
- username: remote_monitoring_user
|
|
|
- password: YOUR_PASSWORD
|
|
|
+ - module: elasticsearch
|
|
|
+ metricsets:
|
|
|
+ - ccr
|
|
|
+ - cluster_stats
|
|
|
+ - index
|
|
|
+ - index_recovery
|
|
|
+ - index_summary
|
|
|
+ - ml_job
|
|
|
+ - node_stats
|
|
|
+ - shard
|
|
|
+ period: 10s
|
|
|
+ hosts: ["http://localhost:9200"]
|
|
|
+ #username: "user"
|
|
|
+ #password: "secret"
|
|
|
+ xpack.enabled: true
|
|
|
----------------------------------
|
|
|
---
|
|
|
|
|
|
-.. If you configured {es} to use <<configuring-tls,encrypted communications>>,
|
|
|
-you must access it via HTTPS. For example, use a `hosts` setting like
|
|
|
-`https://localhost:9200` in the `modules.d/elasticsearch-xpack.yml` file.
|
|
|
+By default, the module collects {es} monitoring metrics from
|
|
|
+`http://localhost:9200`. If that host and port number are not correct, you must
|
|
|
+update the `hosts` setting. If you configured {es} to use encrypted
|
|
|
+communications, you must access it via HTTPS. For example, use a `hosts` setting
|
|
|
+like `https://localhost:9200`.
|
|
|
+// end::configure-es-module[]
|
|
|
|
|
|
-.. Identify where to send the monitoring data. +
|
|
|
+// tag::remote-monitoring-user[]
|
|
|
+If Elastic {security-features} are enabled, you must also provide a user ID
|
|
|
+and password so that {metricbeat} can collect metrics successfully:
|
|
|
+
|
|
|
+.. Create a user on the production cluster that has the
|
|
|
+{stack-ov}/built-in-roles.html[`remote_monitoring_collector` built-in role].
|
|
|
+Alternatively, use the
|
|
|
+{stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user].
|
|
|
+
|
|
|
+.. Add the `username` and `password` settings to the {es} module configuration
|
|
|
+file.
|
|
|
+// end::remote-monitoring-user[]
|
|
|
+--
|
|
|
+
|
|
|
+. Identify where to send the monitoring data. +
|
|
|
+
|
|
|
--
|
|
|
TIP: In production environments, we strongly recommend using a separate cluster
|
|
@@ -136,48 +156,43 @@ configuration file (`metricbeat.yml`):
|
|
|
[source,yaml]
|
|
|
----------------------------------
|
|
|
output.elasticsearch:
|
|
|
+ # Array of hosts to connect to.
|
|
|
hosts: ["http://es-mon-1:9200", "http://es-mon2:9200"] <1>
|
|
|
+
|
|
|
+ # Optional protocol and basic auth credentials.
|
|
|
+ #protocol: "https"
|
|
|
+ #username: "elastic"
|
|
|
+ #password: "changeme"
|
|
|
----------------------------------
|
|
|
<1> In this example, the data is stored on a monitoring cluster with nodes
|
|
|
`es-mon-1` and `es-mon-2`.
|
|
|
|
|
|
+If you configured the monitoring cluster to use encrypted communications, you
|
|
|
+must access it via HTTPS. For example, use a `hosts` setting like
|
|
|
+`https://es-mon-1:9200`.
|
|
|
+
|
|
|
IMPORTANT: The {es} {monitor-features} use ingest pipelines, therefore the
|
|
|
cluster that stores the monitoring data must have at least one
|
|
|
<<ingest,ingest node>>.
|
|
|
|
|
|
-For more information about these configuration options, see
|
|
|
-{metricbeat-ref}/elasticsearch-output.html[Configure the {es} output].
|
|
|
---
|
|
|
-
|
|
|
-.. If {es} {security-features} are enabled on the monitoring cluster, you
|
|
|
-must provide a valid user ID and password so that {metricbeat} can send metrics
|
|
|
-successfully.
|
|
|
+If {es} {security-features} are enabled on the monitoring cluster, you must
|
|
|
+provide a valid user ID and password so that {metricbeat} can send metrics
|
|
|
+successfully:
|
|
|
|
|
|
-... Create a user on the monitoring cluster that has the
|
|
|
+.. Create a user on the monitoring cluster that has the
|
|
|
{stack-ov}/built-in-roles.html[`remote_monitoring_agent` built-in role].
|
|
|
Alternatively, use the
|
|
|
{stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user].
|
|
|
|
|
|
-... Add the `username` and `password` settings to the {es} output information in
|
|
|
-the {metricbeat} configuration file (`metricbeat.yml`):
|
|
|
-+
|
|
|
---
|
|
|
-[source,yaml]
|
|
|
-----------------------------------
|
|
|
-output.elasticsearch:
|
|
|
- ...
|
|
|
- username: remote_monitoring_user
|
|
|
- password: YOUR_PASSWORD
|
|
|
-----------------------------------
|
|
|
---
|
|
|
+.. Add the `username` and `password` settings to the {es} output information in
|
|
|
+the {metricbeat} configuration file.
|
|
|
|
|
|
-.. If you configured the monitoring cluster to use
|
|
|
-<<configuring-tls,encrypted communications>>, you must access it via
|
|
|
-HTTPS. For example, use a `hosts` setting like `https://es-mon-1:9200` in the
|
|
|
-`metricbeat.yml` file.
|
|
|
+For more information about these configuration options, see
|
|
|
+{metricbeat-ref}/elasticsearch-output.html[Configure the {es} output].
|
|
|
+--
|
|
|
|
|
|
-. <<starting-elasticsearch,Start {es}>>.
|
|
|
+. <<starting-elasticsearch,Start {es}>> on each node.
|
|
|
|
|
|
-. {metricbeat-ref}/metricbeat-starting.html[Start {metricbeat}].
|
|
|
+. {metricbeat-ref}/metricbeat-starting.html[Start {metricbeat}] on each node.
|
|
|
|
|
|
. {kibana-ref}/monitoring-data.html[View the monitoring data in {kib}].
|