elasticsearch/docs/reference/ml/anomaly-detection/apis/get-job-stats.asciidoc

[role="xpack"]
[testenv="platinum"]
[[ml-get-job-stats]]
=== Get {anomaly-job} statistics API
++++
<titleabbrev>Get job statistics</titleabbrev>
++++

Retrieves usage information for {anomaly-jobs}.

[[ml-get-job-stats-request]]
==== {api-request-title}

`GET _ml/anomaly_detectors/<job_id>/_stats`

`GET _ml/anomaly_detectors/<job_id>,<job_id>/_stats` +

`GET _ml/anomaly_detectors/_stats` +

`GET _ml/anomaly_detectors/_all/_stats` 

[[ml-get-job-stats-prereqs]]
==== {api-prereq-title}

* If the {es} {security-features} are enabled, you must have `monitor_ml`,
`monitor`, `manage_ml`, or `manage` cluster privileges to use this API. See
<<security-privileges>>.

[[ml-get-job-stats-desc]]
==== {api-description-title}

You can get statistics for multiple {anomaly-jobs} in a single API request by
using a group name, a comma-separated list of jobs, or a wildcard expression.
You can get statistics for all {anomaly-jobs} by using `_all`, by specifying `*`
as the `<job_id>`, or by omitting the `<job_id>`.

IMPORTANT: This API returns a maximum of 10,000 jobs.

[[ml-get-job-stats-path-parms]]
==== {api-path-parms-title}

`<job_id>`::
(Optional, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection-default]

[[ml-get-job-stats-query-parms]]
==== {api-query-parms-title}

`allow_no_jobs`::
(Optional, boolean)
include::{docdir}/ml/ml-shared.asciidoc[tag=allow-no-jobs]

[[ml-get-job-stats-results]]
==== {api-response-body-title}

The API returns the following information about the operational progress of a
job:

`assignment_explanation`::
(string)
include::{docdir}/ml/ml-shared.asciidoc[tag=assignment-explanation-anomaly-jobs]

[[datacounts]]`data_counts`::
(object) An object that describes the quantity of input to the job and any
related error counts. The `data_count` values are cumulative for the lifetime of
a job. If a model snapshot is reverted or old results are deleted, the job
counts are not reset.

`data_counts`.`bucket_count`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-count-anomaly-jobs]

`data_counts`.`earliest_record_timestamp`:::
(date)
include::{docdir}/ml/ml-shared.asciidoc[tag=earliest-record-timestamp]

`data_counts`.`empty_bucket_count`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=empty-bucket-count]

`data_counts`.`input_bytes`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=input-bytes]

`data_counts`.`input_field_count`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=input-field-count]

`data_counts`.`input_record_count`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=input-record-count]

`data_counts`.`invalid_date_count`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=invalid-date-count]

`data_counts`.`job_id`:::
(string)
include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection]

`data_counts`.`last_data_time`:::
(date)
include::{docdir}/ml/ml-shared.asciidoc[tag=last-data-time]

`data_counts`.`latest_empty_bucket_timestamp`:::
(date)
include::{docdir}/ml/ml-shared.asciidoc[tag=latest-empty-bucket-timestamp]

`data_counts`.`latest_record_timestamp`:::
(date)
include::{docdir}/ml/ml-shared.asciidoc[tag=latest-record-timestamp]

`data_counts`.`latest_sparse_bucket_timestamp`:::
(date)
include::{docdir}/ml/ml-shared.asciidoc[tag=latest-sparse-record-timestamp]

`data_counts`.`missing_field_count`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=missing-field-count]
+
The value of `processed_record_count` includes this count.

`data_counts`.`out_of_order_timestamp_count`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=out-of-order-timestamp-count]

`data_counts`.`processed_field_count`:::
include::{docdir}/ml/ml-shared.asciidoc[tag=processed-field-count]

`data_counts`.`processed_record_count`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=processed-record-count]

`data_counts`.`sparse_bucket_count`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=sparse-bucket-count]

`deleting`::
(boolean)
Indicates that the process of deleting the job is in progress but not yet 
completed. It is only reported when `true`.

[[forecastsstats]]`forecasts_stats`::
(object) An object that provides statistical information about forecasts 
belonging to this job. Some statistics are omitted if no forecasts have been 
made. It has the following properties:
+
--
NOTE: Unless there is at least one forecast, `memory_bytes`, `records`,
`processing_time_ms` and `status` properties are omitted.

--

`forecasts_stats`.`forecasted_jobs`:::
(long) A value of `0` indicates that forecasts do not exist for this job. A 
value of `1` indicates that at least one forecast exists.

`forecasts_stats`.`memory_bytes`:::
(object) The `avg`, `min`, `max` and `total` memory usage in bytes for forecasts 
related to this job. If there are no forecasts, this property is omitted.

`forecasts_stats`.`records`:::
(object) The `avg`, `min`, `max` and `total` number of `model_forecast` documents 
written for forecasts related to this job. If there are no forecasts, this
property is omitted.

`forecasts_stats`.`processing_time_ms`:::
(object) The `avg`, `min`, `max` and `total` runtime in milliseconds for 
forecasts related to this job. If there are no forecasts, this property is omitted.

`forecasts_stats`.`status`:::
(object) The count of forecasts by their status. For example: 
{"finished" : 2, "started" : 1}. If there are no forecasts, this property is omitted.

`forecasts_stats`.`total`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=forecast-total]

`job_id`::
(string)
include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection]

[[modelsizestats]]`model_size_stats`::
(object) An object that provides information about the size and contents of the
model. It has the following properties:
 
`model_size_stats`.`bucket_allocation_failures_count`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-allocation-failures-count]

`model_size_stats`.`categorized_doc_count`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=categorized-doc-count]

`model_size_stats`.`categorization_status`:::
(string)
include::{docdir}/ml/ml-shared.asciidoc[tag=categorization-status]

`model_size_stats`.`dead_category_count`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=dead-category-count]

`model_size_stats`.`frequent_category_count`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=frequent-category-count]

`model_size_stats`.`job_id`:::
(string)
include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection]

`model_size_stats`.`log_time`:::
(date) The timestamp of the `model_size_stats` according to server time.

`model_size_stats`.`memory_status`:::
(string)
include::{docdir}/ml/ml-shared.asciidoc[tag=model-memory-status]

`model_size_stats`.`model_bytes`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=model-bytes]

`model_size_stats`.`model_bytes_exceeded`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=model-bytes-exceeded]

`model_size_stats`.`model_bytes_memory_limit`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=model-memory-limit-anomaly-jobs]

`model_size_stats`.`rare_category_count`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=rare-category-count]

`model_size_stats`.`result_type`:::
(string) For internal use. The type of result.

`model_size_stats`.`total_by_field_count`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=total-by-field-count]

`model_size_stats`.`total_category_count`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=total-category-count]

`model_size_stats`.`total_over_field_count`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=total-over-field-count]

`model_size_stats`.`total_partition_field_count`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=total-partition-field-count]

`model_size_stats`.`timestamp`:::
(date)
include::{docdir}/ml/ml-shared.asciidoc[tag=model-timestamp]

[[stats-node]]`node`::
(object) Contains properties for the node that runs the job. This information is
available only for open jobs.

`node`.`attributes`:::
(object) Lists node attributes. For example,
`{"ml.machine_memory": "17179869184", "ml.max_open_jobs" : "20"}`.
  
`node`.`ephemeral_id`:::
(string)
include::{docdir}/ml/ml-shared.asciidoc[tag=node-ephemeral-id]

`node`.`id`:::
(string)
include::{docdir}/ml/ml-shared.asciidoc[tag=node-id]

`node`.`name`:::
(string) The node name.

`node`.`transport_address`:::
(string) The host and port where transport HTTP connections are accepted.

`open_time`::
(string)
include::{docdir}/ml/ml-shared.asciidoc[tag=open-time]

`state`::
(string)
include::{docdir}/ml/ml-shared.asciidoc[tag=state-anomaly-job]

[[timingstats]]`timing_stats`::
(object) An object that provides statistical information about timing aspect of
this job. It has the following properties:

`timing_stats`.`average_bucket_processing_time_ms`:::
(double) Average of all bucket processing times in milliseconds.

`timing_stats`.`bucket_count`:::
(long)
include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-count]

`timing_stats`.`exponential_average_bucket_processing_time_ms`:::
(double)
include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-time-exponential-average]

`timing_stats`.`exponential_average_bucket_processing_time_per_hour_ms`:::
(double)
include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-time-exponential-average-hour]

`timing_stats`.`job_id`:::
(string)
include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection]

`timing_stats`.`maximum_bucket_processing_time_ms`:::
(double)
include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-time-maximum]

`timing_stats`.`minimum_bucket_processing_time_ms`:::
(double)
include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-time-minimum]

`timing_stats`.`total_bucket_processing_time_ms`:::
(double)
include::{docdir}/ml/ml-shared.asciidoc[tag=bucket-time-total]

[[ml-get-job-stats-response-codes]]
==== {api-response-codes-title}

`404` (Missing resources)::
  If `allow_no_jobs` is `false`, this code indicates that there are no 
  resources that match the request or only partial matches for the request.

[[ml-get-job-stats-example]]
==== {api-examples-title}

[source,console]
--------------------------------------------------
GET _ml/anomaly_detectors/low_request_rate/_stats
--------------------------------------------------
// TEST[skip:Kibana sample data]

The API returns the following results:
[source,js]
----
{
  "count" : 1,
  "jobs" : [
    {
      "job_id" : "low_request_rate",
      "data_counts" : {
        "job_id" : "low_request_rate",
        "processed_record_count" : 1216,
        "processed_field_count" : 1216,
        "input_bytes" : 51678,
        "input_field_count" : 1216,
        "invalid_date_count" : 0,
        "missing_field_count" : 0,
        "out_of_order_timestamp_count" : 0,
        "empty_bucket_count" : 242,
        "sparse_bucket_count" : 0,
        "bucket_count" : 1457,
        "earliest_record_timestamp" : 1575172659612,
        "latest_record_timestamp" : 1580417369440,
        "last_data_time" : 1576017595046,
        "latest_empty_bucket_timestamp" : 1580356800000,
        "input_record_count" : 1216
      },
      "model_size_stats" : {
        "job_id" : "low_request_rate",
        "result_type" : "model_size_stats",
        "model_bytes" : 41480,
        "model_bytes_exceeded" : 0,
        "model_bytes_memory_limit" : 10485760,
        "total_by_field_count" : 3,
        "total_over_field_count" : 0,
        "total_partition_field_count" : 2,
        "bucket_allocation_failures_count" : 0,
        "memory_status" : "ok",
        "categorized_doc_count" : 0,
        "total_category_count" : 0,
        "frequent_category_count" : 0,
        "rare_category_count" : 0,
        "dead_category_count" : 0,
        "categorization_status" : "ok",
        "log_time" : 1576017596000,
        "timestamp" : 1580410800000
      },
      "forecasts_stats" : {
        "total" : 1,
        "forecasted_jobs" : 1,
        "memory_bytes" : {
          "total" : 9179.0,
          "min" : 9179.0,
          "avg" : 9179.0,
          "max" : 9179.0
        },
        "records" : {
          "total" : 168.0,
          "min" : 168.0,
          "avg" : 168.0,
          "max" : 168.0
        },
        "processing_time_ms" : {
          "total" : 40.0,
          "min" : 40.0,
          "avg" : 40.0,
          "max" : 40.0
        },
        "status" : {
          "finished" : 1
        }
      },
      "state" : "opened",
      "node" : {
        "id" : "7bmMXyWCRs-TuPfGJJ_yMw",
        "name" : "node-0",
        "ephemeral_id" : "hoXMLZB0RWKfR9UPPUCxXX",
        "transport_address" : "127.0.0.1:9300",
        "attributes" : {
          "ml.machine_memory" : "17179869184",
          "xpack.installed" : "true",
          "ml.max_open_jobs" : "20"
        }
      },
      "assignment_explanation" : "",
      "open_time" : "13s",
      "timing_stats" : {
        "job_id" : "low_request_rate",
        "bucket_count" : 1457,
        "total_bucket_processing_time_ms" : 1094.000000000001,
        "minimum_bucket_processing_time_ms" : 0.0,
        "maximum_bucket_processing_time_ms" : 48.0,
        "average_bucket_processing_time_ms" : 0.75085792724777,
        "exponential_average_bucket_processing_time_ms" : 0.5571716855800993,
        "exponential_average_bucket_processing_time_per_hour_ms" : 15.0
      }
    }
  ]
}
----