high-cpu-usage.asciidoc 2.9 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100
  1. [[high-cpu-usage]]
  2. === High CPU usage
  3. {es} uses <<modules-threadpool,thread pools>> to manage CPU resources for
  4. concurrent operations. High CPU usage typically means one or more thread pools
  5. are running low.
  6. If a thread pool is depleted, {es} will <<rejected-requests,reject requests>>
  7. related to the thread pool. For example, if the `search` thread pool is
  8. depleted, {es} will reject search requests until more threads are available.
  9. [discrete]
  10. [[diagnose-high-cpu-usage]]
  11. ==== Diagnose high CPU usage
  12. **Check CPU usage**
  13. include::{es-repo-dir}/tab-widgets/cpu-usage-widget.asciidoc[]
  14. **Check hot threads**
  15. If a node has high CPU usage, use the <<cluster-nodes-hot-threads,nodes hot
  16. threads API>> to check for resource-intensive threads running on the node.
  17. [source,console]
  18. ----
  19. GET _nodes/my-node,my-other-node/hot_threads
  20. ----
  21. // TEST[s/\/my-node,my-other-node//]
  22. This API returns a breakdown of any hot threads in plain text.
  23. [discrete]
  24. [[reduce-cpu-usage]]
  25. ==== Reduce CPU usage
  26. The following tips outline the most common causes of high CPU usage and their
  27. solutions.
  28. **Scale your cluster**
  29. Heavy indexing and search loads can deplete smaller thread pools. To better
  30. handle heavy workloads, add more nodes to your cluster or upgrade your existing
  31. nodes to increase capacity.
  32. **Spread out bulk requests**
  33. While more efficient than individual requests, large <<docs-bulk,bulk indexing>>
  34. or <<search-multi-search,multi-search>> requests still require CPU resources. If
  35. possible, submit smaller requests and allow more time between them.
  36. **Cancel long-running searches**
  37. Long-running searches can block threads in the `search` thread pool. To check
  38. for these searches, use the <<tasks,task management API>>.
  39. [source,console]
  40. ----
  41. GET _tasks?actions=*search&detailed
  42. ----
  43. The response's `description` contains the search request and its queries.
  44. `running_time_in_nanos` shows how long the search has been running.
  45. [source,console-result]
  46. ----
  47. {
  48. "nodes" : {
  49. "oTUltX4IQMOUUVeiohTt8A" : {
  50. "name" : "my-node",
  51. "transport_address" : "127.0.0.1:9300",
  52. "host" : "127.0.0.1",
  53. "ip" : "127.0.0.1:9300",
  54. "tasks" : {
  55. "oTUltX4IQMOUUVeiohTt8A:464" : {
  56. "node" : "oTUltX4IQMOUUVeiohTt8A",
  57. "id" : 464,
  58. "type" : "transport",
  59. "action" : "indices:data/read/search",
  60. "description" : "indices[my-index], search_type[QUERY_THEN_FETCH], source[{\"query\":...}]",
  61. "start_time_in_millis" : 4081771730000,
  62. "running_time_in_nanos" : 13991383,
  63. "cancellable" : true
  64. }
  65. }
  66. }
  67. }
  68. }
  69. ----
  70. // TESTRESPONSE[skip: no way to get tasks]
  71. To cancel a search and free up resources, use the API's `_cancel` endpoint.
  72. [source,console]
  73. ----
  74. POST _tasks/oTUltX4IQMOUUVeiohTt8A:464/_cancel
  75. ----
  76. For additional tips on how to track and avoid resource-intensive searches, see
  77. <<avoid-expensive-searches,Avoid expensive searches>>.