elasticsearch/docs/reference/modules/indices/circuit_breaker.asciidoc

[[circuit-breaker]]
=== Circuit breaker settings
[[circuit-breaker-description]]
// tag::circuit-breaker-description-tag[]
{es} contains multiple circuit breakers used to prevent operations from causing an OutOfMemoryError. Each breaker specifies a limit for how much memory it can use. Additionally, there is a parent-level breaker that specifies the total amount of memory that can be used across all breakers.

Except where noted otherwise, these settings can be dynamically updated on a
live cluster with the <<cluster-update-settings,cluster-update-settings>> API.
// end::circuit-breaker-description-tag[]

For information about circuit breaker errors, see <<circuit-breaker-errors>>.

[[parent-circuit-breaker]]
[discrete]
==== Parent circuit breaker

The parent-level breaker can be configured with the following settings:

`indices.breaker.total.use_real_memory`::
    (<<static-cluster-setting,Static>>)
    Determines whether the parent breaker should take real
    memory usage into account (`true`) or only consider the amount that is
    reserved by child circuit breakers (`false`). Defaults to `true`.

[[indices-breaker-total-limit]]
// tag::indices-breaker-total-limit-tag[]
`indices.breaker.total.limit` {ess-icon}::
    (<<dynamic-cluster-setting,Dynamic>>)
    Starting limit for overall parent breaker. Defaults to 70% of JVM heap if
    `indices.breaker.total.use_real_memory` is `false`. If `indices.breaker.total.use_real_memory`
    is `true`, defaults to 95% of the JVM heap.
// end::indices-breaker-total-limit-tag[]

[[fielddata-circuit-breaker]]
[discrete]
==== Field data circuit breaker
The field data circuit breaker estimates the heap memory required to load a
field into the <<modules-fielddata,field data cache>>. If loading the field would
cause the cache to exceed a predefined memory limit, the circuit breaker stops the
operation and returns an error.

[[fielddata-circuit-breaker-limit]]
// tag::fielddata-circuit-breaker-limit-tag[]
`indices.breaker.fielddata.limit` {ess-icon}::
    (<<dynamic-cluster-setting,Dynamic>>)
    Limit for fielddata breaker. Defaults to 40% of JVM heap.
// end::fielddata-circuit-breaker-limit-tag[]

[[fielddata-circuit-breaker-overhead]]
// tag::fielddata-circuit-breaker-overhead-tag[]
`indices.breaker.fielddata.overhead` {ess-icon}::
    (<<dynamic-cluster-setting,Dynamic>>)
    A constant that all field data estimations are multiplied with to determine a
    final estimation. Defaults to `1.03`.
// end::fielddata-circuit-breaker-overhead-tag[]

[[request-circuit-breaker]]
[discrete]
==== Request circuit breaker

The request circuit breaker allows Elasticsearch to prevent per-request data
structures (for example, memory used for calculating aggregations during a
request) from exceeding a certain amount of memory.

[[request-breaker-limit]]
// tag::request-breaker-limit-tag[]
`indices.breaker.request.limit` {ess-icon}::
    (<<dynamic-cluster-setting,Dynamic>>)
    Limit for request breaker, defaults to 60% of JVM heap.
// end::request-breaker-limit-tag[]

[[request-breaker-overhead]]
// tag::request-breaker-overhead-tag[]
`indices.breaker.request.overhead` {ess-icon}::
    (<<dynamic-cluster-setting,Dynamic>>)
    A constant that all request estimations are multiplied with to determine a
    final estimation. Defaults to `1`.
// end::request-breaker-overhead-tag[]

[[in-flight-circuit-breaker]]
[discrete]
==== In flight requests circuit breaker

The in flight requests circuit breaker allows Elasticsearch to limit the memory usage of all
currently active incoming requests on transport or HTTP level from exceeding a certain amount of
memory on a node. The memory usage is based on the content length of the request itself. This
circuit breaker also considers that memory is not only needed for representing the raw request but
also as a structured object which is reflected by default overhead.

`network.breaker.inflight_requests.limit`::
    (<<dynamic-cluster-setting,Dynamic>>)
    Limit for in flight requests breaker, defaults to 100% of JVM heap. This means that it is bound
    by the limit configured for the parent circuit breaker.

`network.breaker.inflight_requests.overhead`::
    (<<dynamic-cluster-setting,Dynamic>>)
    A constant that all in flight requests estimations are multiplied with to determine a
    final estimation. Defaults to 2.

[[accounting-circuit-breaker]]
[discrete]
==== Accounting requests circuit breaker

The accounting circuit breaker allows Elasticsearch to limit the memory
usage of things held in memory that are not released when a request is
completed. This includes things like the Lucene segment memory.

`indices.breaker.accounting.limit`::
    (<<dynamic-cluster-setting,Dynamic>>)
    Limit for accounting breaker, defaults to 100% of JVM heap. This means that it is bound
    by the limit configured for the parent circuit breaker.

`indices.breaker.accounting.overhead`::
    (<<dynamic-cluster-setting,Dynamic>>)
    A constant that all accounting estimations are multiplied with to determine a
    final estimation. Defaults to 1

[[script-compilation-circuit-breaker]]
[discrete]
==== Script compilation circuit breaker

Slightly different than the previous memory-based circuit breaker, the script
compilation circuit breaker limits the number of inline script compilations
within a period of time.

See the "prefer-parameters" section of the <<modules-scripting-using,scripting>>
documentation for more information.

`script.max_compilations_rate`::
    (<<dynamic-cluster-setting,Dynamic>>)
    Limit for the number of unique dynamic scripts within a certain interval
    that are allowed to be compiled. Defaults to `150/5m`,
    meaning 150 every 5 minutes.

If the cluster regularly hits the given `max_compilation_rate`, it's possible the
script cache is undersized, use <<cluster-nodes-stats,Nodes Stats>> to inspect
the number of recent cache evictions, `script.cache_evictions_history` and
compilations `script.compilations_history`.  If there are a large
number of recent cache evictions or compilations, the script cache may be
undersized, consider doubling the size of the script cache via the setting
`script.cache.max_size`.

[[regex-circuit-breaker]]
[discrete]
==== Regex circuit breaker

Poorly written regular expressions can degrade cluster stability and
performance. The regex circuit breaker limits the use and complexity of
{painless}/painless-regexes.html[regex in Painless scripts].

[[script-painless-regex-enabled]]
`script.painless.regex.enabled`::
(<<static-cluster-setting,Static>>) Enables regex in Painless scripts. Accepts:

`limited` (Default):::
Enables regex but limits complexity using the
<<script-painless-regex-limit-factor,`script.painless.regex.limit-factor`>>
cluster setting.

`true`:::
Enables regex with no complexity limits. Disables the regex circuit breaker.

`false`:::
Disables regex. Any Painless script containing a regular expression returns an
error.

[[script-painless-regex-limit-factor]]
`script.painless.regex.limit-factor`::
(<<static-cluster-setting,Static>>) Limits the number of characters a regular
expression in a Painless script can consider. {es} calculates this limit by
multiplying the setting value by the script input's character length.
+
For example, the input `foobarbaz` has a character length of `9`. If
`script.painless.regex.limit-factor` is `6`, a regular expression on `foobarbaz`
can consider up to 54 (9 * 6) characters. If the expression exceeds this limit,
it triggers the regex circuit breaker and returns an error.
+
{es} only applies this limit if
<<script-painless-regex-enabled,`script.painless.regex.enabled`>> is `limited`.

[[circuit-breakers-page-eql]]
[discrete]
=== EQL circuit breaker

When a <<eql-sequences, sequence>> query is executed, the node handling the query
needs to keep some structures in memory, which are needed by the algorithm
implementing the sequence matching. When large amounts of data need to be processed,
and/or a large amount of matched sequences is requested by the user (by setting the
<<eql-search-api-params-size, size>> query param), the memory occupied by those
structures could potentially exceed the available memory of the JVM. This would cause
an `OutOfMemory` exception which would bring down the node.

To prevent this from happening, a special <<circuit-breaker, circuit breaker>> is used,
which limits the memory allocation during the execution of a <<eql-sequences, sequence>>
query. When the breaker is triggered, an `org.elasticsearch.common.breaker.CircuitBreakingException`
is thrown and a descriptive error message is returned to the user.

This <<circuit-breaker, circuit breaker>> can be configured using the following settings:

`breaker.eql_sequence.limit`::
(<<cluster-update-settings,Dynamic>>) The limit for circuit breaker used to restrict
the memory utilisation during the execution of an EQL sequence query. This value is
defined as a percentage of the JVM heap. Defaults to `50%`. If the
<<parent-circuit-breaker,parent circuit breaker>> is set to a value less than `50%`,
this setting uses that value as its default instead.

`breaker.eql_sequence.overhead`::
(<<cluster-update-settings,Dynamic>>) A constant that sequence query memory
estimates are multiplied by to determine a final estimate. Defaults to `1`.

`breaker.eql_sequence.type`::
(<<static-cluster-setting,Static>>) Circuit breaker type. Valid values are:

`memory` (Default):::
The breaker limits memory usage for EQL sequence queries.

`noop`:::
Disables the breaker.

[[circuit-breakers-page-model-inference]]
[discrete]
==== {ml-cap} circuit breaker

`breaker.model_inference.limit`::
(<<cluster-update-settings,Dynamic>>) The limit for the trained model circuit
breaker. This value is defined as a percentage of the JVM heap. Defaults to
`50%`. If the <<parent-circuit-breaker,parent circuit breaker>> is set to a
value less than `50%`, this setting uses that value as its default instead.

`breaker.model_inference.overhead`::
(<<cluster-update-settings,Dynamic>>) A constant that all trained model
estimations are multiplied by to determine a final estimation. See
<<circuit-breaker>>. Defaults to `1`.

`breaker.model_inference.type`::
(<<static-cluster-setting,Static>>) The underlying type of the circuit breaker.
There are two valid options: `noop` and `memory`. `noop` means the circuit
breaker does nothing to prevent too much memory usage. `memory` means the
circuit breaker tracks the memory used by trained models and can potentially
break and prevent `OutOfMemory` errors. The default value is `memory`.