local-export.asciidoc 4.5 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596
  1. [role="xpack"]
  2. [[local-exporter]]
  3. === Local exporters
  4. include::production.asciidoc[tag=monitoring-rec]
  5. The `local` exporter is the default exporter in {monitoring}. It routes data
  6. back into the same (local) cluster. In other words, it uses the production
  7. cluster as the monitoring cluster. For example:
  8. [source,yaml]
  9. ---------------------------------------------------
  10. xpack.monitoring.exporters.my_local_exporter: <1>
  11. type: local
  12. ---------------------------------------------------
  13. <1> The exporter name uniquely defines the exporter, but it is otherwise unused.
  14. This exporter exists to provide a convenient option when hardware is simply not
  15. available. It is also a way for developers to get an idea of what their actions
  16. do for pre-production clusters when they do not have the time or resources to
  17. provide a separate monitoring cluster. However, this exporter has disadvantages
  18. that impact the local cluster:
  19. * All indexing impacts the local cluster and the nodes that hold the monitoring
  20. indices' shards.
  21. * Most collectors run on the elected master node. Therefore most indexing occurs
  22. with the elected master node as the coordinating node, which is a bad practice.
  23. * Any usage of {monitoring} for {kib} uses the local cluster's resources for
  24. searches and aggregations, which means that they might not be available for
  25. non-monitoring tasks.
  26. * If the local cluster goes down, the monitoring cluster has inherently gone
  27. down with it (and vice versa), which generally defeats the purpose of monitoring.
  28. For the `local` exporter, all setup occurs only on the elected master node. This
  29. means that if you do not see any monitoring templates or ingest pipelines, the
  30. elected master node is having issues or it is not configured in the same way.
  31. Unlike the `http` exporter, the `local` exporter has the advantage of accessing
  32. the monitoring cluster's up-to-date cluster state. It can therefore always check
  33. that the templates and ingest pipelines exist without a performance penalty. If
  34. the elected master node encounters errors while trying to create the monitoring
  35. resources, it logs errors, ignores that collection, and tries again after the
  36. next collection.
  37. The elected master node is the only node to set up resources for the `local`
  38. exporter. Therefore all other nodes wait for the resources to be set up before
  39. indexing any monitoring data from their own collectors. Each of these nodes logs
  40. a message indicating that they are waiting for the resources to be set up.
  41. One benefit of the `local` exporter is that it lives within the cluster and
  42. therefore no extra configuration is required when the cluster is secured with
  43. {stack} {security-features}. All operations, including indexing operations, that
  44. occur from a `local` exporter make use of the internal transport mechanisms
  45. within {es}. This behavior enables the exporter to be used without providing any
  46. user credentials when {security-features} are enabled.
  47. For more information about the configuration options for the `local` exporter,
  48. see <<local-exporter-settings>>.
  49. [[local-exporter-cleaner]]
  50. ==== Cleaner service
  51. One feature of the `local` exporter, which is not present in the `http` exporter,
  52. is a cleaner service. The cleaner service runs once per day at 01:00 AM UTC on
  53. the elected master node.
  54. The role of the cleaner service is to clean, or curate, the monitoring indices
  55. that are older than a configurable amount of time (the default is `7d`). This
  56. cleaner exists as part of the `local` exporter as a safety mechanism. The `http`
  57. exporter does not make use of it because it could enable a single misconfigured
  58. node to prematurely curate data from other production clusters that share the
  59. same monitoring cluster.
  60. In a dedicated monitoring cluster, you can use the cleaner service without
  61. having to monitor the monitoring cluster itself. For example:
  62. [source,yaml]
  63. ---------------------------------------------------
  64. xpack.monitoring.collection.enabled: false <1>
  65. xpack.monitoring.history.duration: 3d <2>
  66. ---------------------------------------------------
  67. <1> Disables the collection of data on the monitoring cluster.
  68. <2> Lowers the default history duration from `7d` to `3d`. The minimum value is
  69. `1d`. This setting can be modified only when using a Gold or higher level
  70. license. For the Basic license level, it uses the default of 7 days.
  71. To disable the cleaner service, add a disabled local exporter:
  72. [source,yaml]
  73. ----
  74. xpack.monitoring.exporters.my_local.type: local <1>
  75. xpack.monitoring.exporters.my_local.enabled: false <2>
  76. ----
  77. <1> Adds a local exporter named `my_local`
  78. <2> Disables the local exporter. This also disables the cleaner service.