|
@@ -146,23 +146,22 @@ JVM. If such a process does not connect within the time period specified by this
|
|
|
setting then the process is assumed to have failed. Defaults to `10s`. The minimum
|
|
|
value for this setting is `5s`.
|
|
|
|
|
|
+[float]
|
|
|
[[model-inference-circuit-breaker]]
|
|
|
==== {ml-cap} circuit breaker settings
|
|
|
|
|
|
-`breaker.model_inference.limit` (<<cluster-update-settings,Dynamic>>)
|
|
|
-Limit for model inference breaker, defaults to 50% of JVM heap.
|
|
|
-If the parent circuit breaker is less than 50% of JVM heap, it is bound
|
|
|
-to that limit instead.
|
|
|
-See <<circuit-breaker>>.
|
|
|
-
|
|
|
-`breaker.model_inference.overhead` (<<cluster-update-settings,Dynamic>>)
|
|
|
-A constant that all accounting estimations are multiplied with to determine
|
|
|
-a final estimation. Defaults to 1.
|
|
|
-See <<circuit-breaker>>.
|
|
|
-
|
|
|
-`breaker.model_inference.type`
|
|
|
-The underlying type of the circuit breaker. There are two valid options:
|
|
|
-`noop`, meaning the circuit breaker does nothing to prevent too much memory usage,
|
|
|
-`memory`, meaning the circuit breaker tracks the memory used by inference models and
|
|
|
-could potentially break and prevent OutOfMemory errors.
|
|
|
-The default is `memory`.
|
|
|
+`breaker.model_inference.limit` (<<cluster-update-settings,Dynamic>>)::
|
|
|
+Limit for the model inference breaker, which defaults to 50% of the JVM heap.
|
|
|
+If the parent circuit breaker is less than 50% of the JVM heap, it is bound
|
|
|
+to that limit instead. See <<circuit-breaker>>.
|
|
|
+
|
|
|
+`breaker.model_inference.overhead` (<<cluster-update-settings,Dynamic>>)::
|
|
|
+A constant that all accounting estimations are multiplied by to determine
|
|
|
+a final estimation. Defaults to 1. See <<circuit-breaker>>.
|
|
|
+
|
|
|
+`breaker.model_inference.type`::
|
|
|
+The underlying type of the circuit breaker. There are two valid options: `noop`
|
|
|
+and `memory`. `noop` means the circuit breaker does nothing to prevent too much
|
|
|
+memory usage. `memory` means the circuit breaker tracks the memory used by
|
|
|
+inference models and can potentially break and prevent OutOfMemory errors. The
|
|
|
+default is `memory`.
|