elasticsearch/docs/reference/monitoring/production.asciidoc

[role="xpack"]
[[monitoring-production]]
== Monitoring in a production environment

In production, you should send monitoring data to a separate _monitoring cluster_
so that historical data is available even when the nodes you are monitoring are
not.

// tag::monitoring-rec[]
[IMPORTANT]
=========================
{agent} and {metricbeat} are the recommended methods for collecting and shipping
monitoring data to a monitoring cluster.

If you have previously configured legacy collection methods, you should migrate
to using <<configuring-elastic-agent,{agent}>> or
<<configuring-metricbeat,{metricbeat}>> collection. Do not use legacy collection
alongside other collection methods.
=========================
// end::monitoring-rec[]

If you have at least a Gold Subscription, using a dedicated monitoring cluster
also enables you to monitor multiple clusters from a central location.

To store monitoring data in a separate cluster:

. Set up the {es} cluster you want to use as the monitoring cluster.
For example, you might set up a two host cluster with the nodes `es-mon-1` and
`es-mon-2`.
+
--
[IMPORTANT]
===============================
* Ideally the monitoring cluster and the production cluster run on the same
{stack} version. However, a monitoring cluster on the latest release of
{major-version} also works with production clusters that use the same major
version. Monitoring clusters that use {major-version} also work with production
clusters that use the latest release of {prev-major-version}.
* There must be at least one <<ingest,ingest node>> in the monitoring
cluster; it does not need to be a dedicated ingest node.
===============================
--

.. (Optional) Verify that the collection of monitoring data is disabled on the
monitoring cluster. By default, the `xpack.monitoring.collection.enabled` setting
is `false`.
+
--
For example, you can use the following APIs to review and change this setting:

[source,console]
----------------------------------
GET _cluster/settings

PUT _cluster/settings
{
  "persistent": {
    "xpack.monitoring.collection.enabled": false
  }
}
----------------------------------
// TEST[skip:security errs]
--

.. If the {es} {security-features} are enabled on the monitoring cluster, create
users that can send and retrieve monitoring data:
+
--
NOTE: If you plan to use {kib} to view monitoring data, username and password
credentials must be valid on both the {kib} server and the monitoring cluster.

--

*** If you plan to use {agent},
create a user that has the `remote_monitoring_collector`
<<built-in-roles-remote-monitoring-agent,built-in role>>.

*** If you plan to use {metricbeat},
create a user that has the `remote_monitoring_collector` built-in role and a
user that has the `remote_monitoring_agent`
<<built-in-roles-remote-monitoring-agent,built-in role>>. Alternatively, use the
`remote_monitoring_user` <<built-in-users,built-in user>>.

*** If you plan to use HTTP exporters to route data through your production
cluster, create a user that has the `remote_monitoring_agent`
<<built-in-roles-remote-monitoring-agent,built-in role>>.
+
--
For example, the
following request creates a `remote_monitor` user that has the
`remote_monitoring_agent` role:

[source,console]
---------------------------------------------------------------
POST /_security/user/remote_monitor
{
  "password" : "changeme",
  "roles" : [ "remote_monitoring_agent"],
  "full_name" : "Internal Agent For Remote Monitoring"
}
---------------------------------------------------------------
// TEST[skip:needs-gold+-license]

Alternatively, use the `remote_monitoring_user` <<built-in-users,built-in user>>.
--

. Configure your production cluster to collect data and send it to the
monitoring cluster:
** <<configuring-elastic-agent,{agent} collection methods>>
** <<configuring-metricbeat,{metricbeat} collection methods>>
** <<collecting-monitoring-data,Legacy collection methods>>

. (Optional)
{logstash-ref}/configuring-logstash.html[Configure {ls} to collect data and send it to the monitoring cluster].

. (Optional) {enterprise-search-ref}/monitoring.html[Configure {ents} monitoring].

. (Optional) Configure the {beats} to collect data and send it to the monitoring
cluster. Skip this step for {beats} that are managed by {agent}.
** {auditbeat-ref}/monitoring.html[Auditbeat]
** {filebeat-ref}/monitoring.html[Filebeat]
** {heartbeat-ref}/monitoring.html[Heartbeat]
** {metricbeat-ref}/monitoring.html[Metricbeat]
** {packetbeat-ref}/monitoring.html[Packetbeat]
** {winlogbeat-ref}/monitoring.html[Winlogbeat]

. (Optional) {apm-guide-ref}/monitor-apm.html[Configure APM Server monitoring]

. (Optional) Configure {kib} to collect data and send it to the monitoring cluster:
** {kibana-ref}/monitoring-elastic-agent.html[{agent} collection methods]
** {kibana-ref}/monitoring-metricbeat.html[{metricbeat} collection methods]
** {kibana-ref}/monitoring-kibana.html[Legacy collection methods]

. (Optional) Create a dedicated {kib} instance for monitoring, rather than using
a single {kib} instance to access both your production cluster and monitoring
cluster.
+
--
NOTE: If you log in to {kib} using SAML, Kerberos, PKI, OpenID Connect, or token
authentication providers, a dedicated {kib} instance is *required*. The security
tokens that are used in these contexts are cluster-specific; therefore you
cannot use a single {kib} instance to connect to both production and monitoring
clusters.

--

.. (Optional) Disable the collection of monitoring data in this {kib} instance.
Set the `xpack.monitoring.kibana.collection.enabled` setting to `false` in the
`kibana.yml` file. For more information about this setting, see
{kibana-ref}/monitoring-settings-kb.html[Monitoring settings in {kib}].

. {kibana-ref}/monitoring-data.html[Configure {kib} to retrieve and display the monitoring data].