|
@@ -61,12 +61,12 @@ A {dfeed} resource has the following properties:
|
|
|
|
|
|
`delayed_data_check_config`::
|
|
|
(object) Specifies whether the data feed checks for missing data and
|
|
|
- and the size of the window. For example:
|
|
|
+ the size of the window. For example:
|
|
|
`{"enabled": true, "check_window": "1h"}` See
|
|
|
<<ml-datafeed-delayed-data-check-config>>.
|
|
|
|
|
|
[[ml-datafeed-chunking-config]]
|
|
|
-==== Chunking Configuration Objects
|
|
|
+==== Chunking configuration objects
|
|
|
|
|
|
{dfeeds-cap} might be required to search over long time periods, for several months
|
|
|
or years. This search is split into time chunks in order to ensure the load
|
|
@@ -88,31 +88,33 @@ A chunking configuration object has the following properties:
|
|
|
For example: `3h`.
|
|
|
|
|
|
[[ml-datafeed-delayed-data-check-config]]
|
|
|
-==== Delayed Data Check Configuration Objects
|
|
|
+==== Delayed data check configuration objects
|
|
|
|
|
|
The {dfeed} can optionally search over indices that have already been read in
|
|
|
-an effort to find if any data has since been added to the index. If missing data
|
|
|
-is found, it is a good indication that the `query_delay` option is set too low and
|
|
|
-the data is being indexed after the {dfeed} has passed that moment in time. See
|
|
|
+an effort to determine whether any data has subsequently been added to the index.
|
|
|
+If missing data is found, it is a good indication that the `query_delay` option
|
|
|
+is set too low and the data is being indexed after the {dfeed} has passed that
|
|
|
+moment in time. See
|
|
|
{stack-ov}/ml-delayed-data-detection.html[Working with delayed data].
|
|
|
|
|
|
-This check only runs on real-time {dfeeds}
|
|
|
+This check runs only on real-time {dfeeds}.
|
|
|
|
|
|
The configuration object has the following properties:
|
|
|
|
|
|
`enabled`::
|
|
|
- (boolean) Should the {dfeed} periodically check for data being indexed after reading.
|
|
|
- Defaults to `true`
|
|
|
+ (boolean) Specifies whether the {dfeed} periodically checks for delayed data.
|
|
|
+ Defaults to `true`.
|
|
|
|
|
|
`check_window`::
|
|
|
- (time units) The window of time before the latest finalized bucket that should be searched
|
|
|
- for late data. Defaults to `null` which causes an appropriate `check_window` to be calculated
|
|
|
- when the real-time {dfeed} runs.
|
|
|
- The default `check_window` span calculation is the max between `2h` or `8 * bucket_span`.
|
|
|
+ (time units) The window of time that is searched for late data. This window of
|
|
|
+ time ends with the latest finalized bucket. It defaults to `null`, which
|
|
|
+ causes an appropriate `check_window` to be calculated when the real-time
|
|
|
+ {dfeed} runs. In particular, the default `check_window` span calculation is
|
|
|
+ based on the maximum of `2h` or `8 * bucket_span`.
|
|
|
|
|
|
[float]
|
|
|
[[ml-datafeed-counts]]
|
|
|
-==== {dfeed-cap} Counts
|
|
|
+==== {dfeed-cap} counts
|
|
|
|
|
|
The get {dfeed} statistics API provides information about the operational
|
|
|
progress of a {dfeed}. All of these properties are informational; you cannot
|