|
|
@@ -17,7 +17,6 @@ return an error and the job waits in the `opening` state until sufficient {ml}
|
|
|
node capacity is available.
|
|
|
end::allow-lazy-open[]
|
|
|
|
|
|
-
|
|
|
tag::allow-lazy-start[]
|
|
|
Whether this job should be allowed to start when there is insufficient {ml} node
|
|
|
capacity for it to be immediately assigned to a node. The default is `false`,
|
|
|
@@ -80,71 +79,16 @@ example: `outlier_detection`. See <<ml-dfa-analysis-objects>>.
|
|
|
end::analysis[]
|
|
|
|
|
|
tag::analysis-config[]
|
|
|
-The analysis configuration, which specifies how to analyze the data.
|
|
|
-After you create a job, you cannot change the analysis configuration; all
|
|
|
-the properties are informational. An analysis configuration object has the
|
|
|
-following properties:
|
|
|
-
|
|
|
-`bucket_span`:::
|
|
|
-(<<time-units,time units>>)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-span]
|
|
|
-
|
|
|
-`categorization_field_name`:::
|
|
|
-(string)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=categorization-field-name]
|
|
|
-
|
|
|
-`categorization_filters`:::
|
|
|
-(array of strings)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=categorization-filters]
|
|
|
-
|
|
|
-`categorization_analyzer`:::
|
|
|
-(object or string)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=categorization-analyzer]
|
|
|
-
|
|
|
-`detectors`:::
|
|
|
-(array) An array of detector configuration objects. Detector configuration
|
|
|
-objects specify which data fields a job analyzes. They also specify which
|
|
|
-analytical functions are used. You can specify multiple detectors for a job.
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=detector]
|
|
|
-+
|
|
|
---
|
|
|
-NOTE: If the `detectors` array does not contain at least one detector,
|
|
|
-no analysis can occur and an error is returned.
|
|
|
-
|
|
|
---
|
|
|
-
|
|
|
-`influencers`:::
|
|
|
-(array of strings)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=influencers]
|
|
|
-
|
|
|
-`latency`:::
|
|
|
-(time units)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=latency]
|
|
|
-
|
|
|
-`multivariate_by_fields`:::
|
|
|
-(boolean)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=multivariate-by-fields]
|
|
|
-
|
|
|
-`summary_count_field_name`:::
|
|
|
-(string)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=summary-count-field-name]
|
|
|
-
|
|
|
+The analysis configuration, which specifies how to analyze the data. After you
|
|
|
+create a job, you cannot change the analysis configuration; all the properties
|
|
|
+are informational.
|
|
|
end::analysis-config[]
|
|
|
|
|
|
tag::analysis-limits[]
|
|
|
Limits can be applied for the resources required to hold the mathematical models
|
|
|
in memory. These limits are approximate and can be set per job. They do not
|
|
|
-control the memory used by other processes, for example the {es} Java
|
|
|
-processes. If necessary, you can increase the limits after the job is created.
|
|
|
-The `analysis_limits` object has the following properties:
|
|
|
-
|
|
|
-`categorization_examples_limit`:::
|
|
|
-(long)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=categorization-examples-limit]
|
|
|
-
|
|
|
-`model_memory_limit`:::
|
|
|
-(long or string)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=model-memory-limit]
|
|
|
+control the memory used by other processes, for example the {es} Java processes.
|
|
|
+If necessary, you can increase the limits after the job is created.
|
|
|
end::analysis-limits[]
|
|
|
|
|
|
tag::analyzed-fields[]
|
|
|
@@ -212,15 +156,15 @@ object. If it is a string it must refer to a
|
|
|
is an object it has the following properties:
|
|
|
--
|
|
|
|
|
|
-`char_filter`::::
|
|
|
+`analysis_config`.`categorization_analyzer`.`char_filter`::::
|
|
|
(array of strings or objects)
|
|
|
include::{docdir}/ml/ml-shared.asciidoc[tag=char-filter]
|
|
|
|
|
|
-`tokenizer`::::
|
|
|
+`analysis_config`.`categorization_analyzer`.`tokenizer`::::
|
|
|
(string or object)
|
|
|
include::{docdir}/ml/ml-shared.asciidoc[tag=tokenizer]
|
|
|
|
|
|
-`filter`::::
|
|
|
+`analysis_config`.`categorization_analyzer`.`filter`::::
|
|
|
(array of strings or objects)
|
|
|
include::{docdir}/ml/ml-shared.asciidoc[tag=filter]
|
|
|
end::categorization-analyzer[]
|
|
|
@@ -286,11 +230,11 @@ on {es} is managed. Chunking configuration controls how the size of these time
|
|
|
chunks are calculated and is an advanced configuration option.
|
|
|
A chunking configuration object has the following properties:
|
|
|
|
|
|
-`mode`:::
|
|
|
+`chunking_config`.`mode`:::
|
|
|
(string)
|
|
|
include::{docdir}/ml/ml-shared.asciidoc[tag=mode]
|
|
|
|
|
|
-`time_span`:::
|
|
|
+`chunking_config`.`time_span`:::
|
|
|
(<<time-units,time units>>)
|
|
|
include::{docdir}/ml/ml-shared.asciidoc[tag=time-span]
|
|
|
end::chunking-config[]
|
|
|
@@ -300,11 +244,10 @@ An array of custom rule objects, which enable you to customize the way detectors
|
|
|
operate. For example, a rule may dictate to the detector conditions under which
|
|
|
results should be skipped. For more examples, see
|
|
|
{ml-docs}/ml-configuring-detector-custom-rules.html[Customizing detectors with custom rules].
|
|
|
-A custom rule has the following properties:
|
|
|
-+
|
|
|
---
|
|
|
-`actions`::
|
|
|
-(array) The set of actions to be triggered when the rule applies. If
|
|
|
+end::custom-rules[]
|
|
|
+
|
|
|
+tag::custom-rules-actions[]
|
|
|
+The set of actions to be triggered when the rule applies. If
|
|
|
more than one action is specified the effects of all actions are combined. The
|
|
|
available actions include:
|
|
|
|
|
|
@@ -316,49 +259,47 @@ model. Unless you also specify `skip_result`, the results will be created as
|
|
|
usual. This action is suitable when certain values are expected to be
|
|
|
consistently anomalous and they affect the model in a way that negatively
|
|
|
impacts the rest of the results.
|
|
|
+end::custom-rules-actions[]
|
|
|
|
|
|
-`scope`::
|
|
|
-(object) An optional scope of series where the rule applies. A rule must either
|
|
|
+tag::custom-rules-scope[]
|
|
|
+An optional scope of series where the rule applies. A rule must either
|
|
|
have a non-empty scope or at least one condition. By default, the scope includes
|
|
|
all series. Scoping is allowed for any of the fields that are also specified in
|
|
|
`by_field_name`, `over_field_name`, or `partition_field_name`. To add a scope
|
|
|
for a field, add the field name as a key in the scope object and set its value
|
|
|
to an object with the following properties:
|
|
|
-
|
|
|
-`filter_id`:::
|
|
|
-(string) The id of the filter to be used.
|
|
|
-
|
|
|
-`filter_type`:::
|
|
|
-(string) Either `include` (the rule applies for values in the filter) or
|
|
|
-`exclude` (the rule applies for values not in the filter). Defaults to
|
|
|
-`include`.
|
|
|
-
|
|
|
-`conditions`::
|
|
|
-(array) An optional array of numeric conditions when the rule applies. A rule
|
|
|
-must either have a non-empty scope or at least one condition. Multiple
|
|
|
-conditions are combined together with a logical `AND`. A condition has the
|
|
|
-following properties:
|
|
|
-
|
|
|
-`applies_to`:::
|
|
|
-(string) Specifies the result property to which the condition applies. The
|
|
|
-available options are `actual`, `typical`, `diff_from_typical`, `time`.
|
|
|
-
|
|
|
-`operator`:::
|
|
|
-(string) Specifies the condition operator. The available options are `gt`
|
|
|
-(greater than), `gte` (greater than or equals), `lt` (less than) and `lte` (less
|
|
|
-than or equals).
|
|
|
-
|
|
|
-`value`:::
|
|
|
-(double) The value that is compared against the `applies_to` field using the
|
|
|
-`operator`.
|
|
|
---
|
|
|
-+
|
|
|
---
|
|
|
-NOTE: If your detector uses `lat_long`, `metric`, `rare`, or `freq_rare`
|
|
|
-functions, you can only specify `conditions` that apply to `time`.
|
|
|
-
|
|
|
---
|
|
|
-end::custom-rules[]
|
|
|
+end::custom-rules-scope[]
|
|
|
+
|
|
|
+tag::custom-rules-scope-filter-id[]
|
|
|
+The id of the filter to be used.
|
|
|
+end::custom-rules-scope-filter-id[]
|
|
|
+
|
|
|
+tag::custom-rules-scope-filter-type[]
|
|
|
+Either `include` (the rule applies for values in the filter) or `exclude` (the
|
|
|
+rule applies for values not in the filter). Defaults to `include`.
|
|
|
+end::custom-rules-scope-filter-type[]
|
|
|
+
|
|
|
+tag::custom-rules-conditions[]
|
|
|
+An optional array of numeric conditions when the rule applies. A rule must
|
|
|
+either have a non-empty scope or at least one condition. Multiple conditions are
|
|
|
+combined together with a logical `AND`. A condition has the following properties:
|
|
|
+end::custom-rules-conditions[]
|
|
|
+
|
|
|
+tag::custom-rules-conditions-applies-to[]
|
|
|
+Specifies the result property to which the condition applies. The available
|
|
|
+options are `actual`, `typical`, `diff_from_typical`, `time`. If your detector
|
|
|
+uses `lat_long`, `metric`, `rare`, or `freq_rare` functions, you can only
|
|
|
+specify conditions that apply to `time`.
|
|
|
+end::custom-rules-conditions-applies-to[]
|
|
|
+
|
|
|
+tag::custom-rules-conditions-operator[]
|
|
|
+Specifies the condition operator. The available options are `gt` (greater than),
|
|
|
+`gte` (greater than or equals), `lt` (less than) and `lte` (less than or equals).
|
|
|
+end::custom-rules-conditions-operator[]
|
|
|
+
|
|
|
+tag::custom-rules-conditions-value[]
|
|
|
+The value that is compared against the `applies_to` field using the `operator`.
|
|
|
+end::custom-rules-conditions-value[]
|
|
|
|
|
|
tag::custom-settings[]
|
|
|
Advanced configuration option. Contains custom meta data about the job. For
|
|
|
@@ -375,16 +316,14 @@ a {dfeed}, these properties are automatically set.
|
|
|
When data is received via the <<ml-post-data,post data>> API, it is not stored
|
|
|
in {es}. Only the results for {anomaly-detect} are retained.
|
|
|
|
|
|
-A data description object has the following properties:
|
|
|
-
|
|
|
-`format`:::
|
|
|
+`data_description`.`format`:::
|
|
|
(string) Only `JSON` format is supported at this time.
|
|
|
|
|
|
-`time_field`:::
|
|
|
+`data_description`.`time_field`:::
|
|
|
(string) The name of the field that contains the timestamp.
|
|
|
The default value is `time`.
|
|
|
|
|
|
-`time_format`:::
|
|
|
+`data_description`.`time_format`:::
|
|
|
(string)
|
|
|
include::{docdir}/ml/ml-shared.asciidoc[tag=time-format]
|
|
|
--
|
|
|
@@ -507,13 +446,11 @@ moment in time. See
|
|
|
|
|
|
This check runs only on real-time {dfeeds}.
|
|
|
|
|
|
-The configuration object has the following properties:
|
|
|
-
|
|
|
-`enabled`::
|
|
|
+`delayed_data_check_config`.`enabled`::
|
|
|
(boolean) Specifies whether the {dfeed} periodically checks for delayed data.
|
|
|
Defaults to `true`.
|
|
|
|
|
|
-`check_window`::
|
|
|
+`delayed_data_check_config`.`check_window`::
|
|
|
(<<time-units,time units>>) The window of time that is searched for late data.
|
|
|
This window of time ends with the latest finalized bucket. It defaults to
|
|
|
`null`, which causes an appropriate `check_window` to be calculated when the
|
|
|
@@ -571,51 +508,6 @@ the detectors in the `analysis_config`, starting at zero. You can use this
|
|
|
identifier when you want to update a specific detector.
|
|
|
end::detector-index[]
|
|
|
|
|
|
-tag::detector[]
|
|
|
-A detector has the following properties:
|
|
|
-
|
|
|
-`by_field_name`::::
|
|
|
-(string)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=by-field-name]
|
|
|
-
|
|
|
-`custom_rules`::::
|
|
|
-(array)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=custom-rules]
|
|
|
-
|
|
|
-`detector_description`::::
|
|
|
-(string)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=detector-description]
|
|
|
-
|
|
|
-`detector_index`::::
|
|
|
-(integer)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=detector-index]
|
|
|
-
|
|
|
-`exclude_frequent`::::
|
|
|
-(string)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=exclude-frequent]
|
|
|
-
|
|
|
-`field_name`::::
|
|
|
-(string)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=detector-field-name]
|
|
|
-
|
|
|
-`function`::::
|
|
|
-(string)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=function]
|
|
|
-
|
|
|
-`over_field_name`::::
|
|
|
-(string)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=over-field-name]
|
|
|
-
|
|
|
-`partition_field_name`::::
|
|
|
-(string)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=partition-field-name]
|
|
|
-
|
|
|
-`use_null`::::
|
|
|
-(boolean)
|
|
|
-include::{docdir}/ml/ml-shared.asciidoc[tag=use-null]
|
|
|
-
|
|
|
-end::detector[]
|
|
|
-
|
|
|
tag::eta[]
|
|
|
The shrinkage applied to the weights. Smaller values result
|
|
|
in larger forests which have better generalization error. However, the smaller
|
|
|
@@ -911,22 +803,21 @@ be seen in the model plot.
|
|
|
|
|
|
Model plot config can be configured when the job is created or updated later. It
|
|
|
must be disabled if performance issues are experienced.
|
|
|
-
|
|
|
-The `model_plot_config` object has the following properties:
|
|
|
-
|
|
|
-`enabled`:::
|
|
|
-(boolean) If true, enables calculation and storage of the model bounds for
|
|
|
-each entity that is being analyzed. By default, this is not enabled.
|
|
|
-
|
|
|
-`terms`:::
|
|
|
-experimental[] (string) Limits data collection to this comma separated list of
|
|
|
-partition or by field values. If terms are not specified or it is an empty
|
|
|
-string, no filtering is applied. For example, "CPU,NetworkIn,DiskWrites".
|
|
|
-Wildcards are not supported. Only the specified `terms` can be viewed when
|
|
|
-using the Single Metric Viewer.
|
|
|
--
|
|
|
end::model-plot-config[]
|
|
|
|
|
|
+tag::model-plot-config-enabled[]
|
|
|
+If true, enables calculation and storage of the model bounds for each entity
|
|
|
+that is being analyzed. By default, this is not enabled.
|
|
|
+end::model-plot-config-enabled[]
|
|
|
+
|
|
|
+tag::model-plot-config-terms[]
|
|
|
+Limits data collection to this comma separated list of partition or by field
|
|
|
+values. If terms are not specified or it is an empty string, no filtering is
|
|
|
+applied. For example, "CPU,NetworkIn,DiskWrites". Wildcards are not supported.
|
|
|
+Only the specified `terms` can be viewed when using the Single Metric Viewer.
|
|
|
+end::model-plot-config-terms[]
|
|
|
+
|
|
|
tag::model-snapshot-id[]
|
|
|
A numerical character string that uniquely identifies the model snapshot. For
|
|
|
example, `1575402236000 `.
|
Lisa Cawley