high-jvm-memory-pressure.asciidoc 3.1 KB

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586878889909192939495
  1. [[high-jvm-memory-pressure]]
  2. === High JVM memory pressure
  3. High JVM memory usage can degrade cluster performance and trigger
  4. <<circuit-breaker-errors,circuit breaker errors>>. To prevent this, we recommend
  5. taking steps to reduce memory pressure if a node's JVM memory usage consistently
  6. exceeds 85%.
  7. [discrete]
  8. [[diagnose-high-jvm-memory-pressure]]
  9. ==== Diagnose high JVM memory pressure
  10. **Check JVM memory pressure**
  11. include::{es-repo-dir}/tab-widgets/jvm-memory-pressure-widget.asciidoc[]
  12. **Check garbage collection logs**
  13. As memory usage increases, garbage collection becomes more frequent and takes
  14. longer. You can track the frequency and length of garbage collection events in
  15. <<logging,`elasticsearch.log`>>. For example, the following event states {es}
  16. spent more than 50% (21 seconds) of the last 40 seconds performing garbage
  17. collection.
  18. [source,log]
  19. ----
  20. [timestamp_short_interval_from_last][INFO ][o.e.m.j.JvmGcMonitorService] [node_id] [gc][number] overhead, spent [21s] collecting in the last [40s]
  21. ----
  22. [discrete]
  23. [[reduce-jvm-memory-pressure]]
  24. ==== Reduce JVM memory pressure
  25. **Reduce your shard count**
  26. Every shard uses memory. In most cases, a small set of large shards uses fewer
  27. resources than many small shards. For tips on reducing your shard count, see
  28. <<size-your-shards>>.
  29. [[avoid-expensive-searches]]
  30. **Avoid expensive searches**
  31. Expensive searches can use large amounts of memory. To better track expensive
  32. searches on your cluster, enable <<index-modules-slowlog,slow logs>>.
  33. Expensive searches may have a large <<paginate-search-results,`size` argument>>,
  34. use aggregations with a large number of buckets, or include
  35. <<query-dsl-allow-expensive-queries,expensive queries>>. To prevent expensive
  36. searches, consider the following setting changes:
  37. * Lower the `size` limit using the
  38. <<index-max-result-window,`index.max_result_window`>> index setting.
  39. * Decrease the maximum number of allowed aggregation buckets using the
  40. <<search-settings-max-buckets,search.max_buckets>> cluster setting.
  41. * Disable expensive queries using the
  42. <<query-dsl-allow-expensive-queries,`search.allow_expensive_queries`>> cluster
  43. setting.
  44. [source,console]
  45. ----
  46. PUT _settings
  47. {
  48. "index.max_result_window": 5000
  49. }
  50. PUT _cluster/settings
  51. {
  52. "persistent": {
  53. "search.max_buckets": 20000,
  54. "search.allow_expensive_queries": false
  55. }
  56. }
  57. ----
  58. // TEST[s/^/PUT my-index\n/]
  59. **Prevent mapping explosions**
  60. Defining too many fields or nesting fields too deeply can lead to
  61. <<mapping-limit-settings,mapping explosions>> that use large amounts of memory.
  62. To prevent mapping explosions, use the <<mapping-settings-limit,mapping limit
  63. settings>> to limit the number of field mappings.
  64. **Spread out bulk requests**
  65. While more efficient than individual requests, large <<docs-bulk,bulk indexing>>
  66. or <<search-multi-search,multi-search>> requests can still create high JVM
  67. memory pressure. If possible, submit smaller requests and allow more time
  68. between them.
  69. **Upgrade node memory**
  70. Heavy indexing and search loads can cause high JVM memory pressure. To better
  71. handle heavy workloads, upgrade your nodes to increase their memory capacity.