post-data.asciidoc 3.1 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110
  1. [role="xpack"]
  2. [[ml-post-data]]
  3. === Post Data to Jobs API
  4. ++++
  5. <titleabbrev>Post Data to Jobs</titleabbrev>
  6. ++++
  7. Sends data to an anomaly detection job for analysis.
  8. ==== Request
  9. `POST _xpack/ml/anomaly_detectors/<job_id>/_data`
  10. ==== Description
  11. The job must have a state of `open` to receive and process the data.
  12. The data that you send to the job must use the JSON format. Multiple JSON
  13. documents can be sent, either adjacent with no separator in between them or
  14. whitespace separated. Newline delimited JSON (NDJSON) is a possible whitespace
  15. separated format, and for this the `Content-Type` header should be set to
  16. `application/x-ndjson`.
  17. Upload sizes are limited to the Elasticsearch HTTP receive buffer size
  18. (default 100 Mb). If your data is larger, split it into multiple chunks
  19. and upload each one separately in sequential time order. When running in
  20. real time, it is generally recommended that you perform many small uploads,
  21. rather than queueing data to upload larger files.
  22. When uploading data, check the <<ml-datacounts,job data counts>> for progress.
  23. The following records will not be processed:
  24. * Records not in chronological order and outside the latency window
  25. * Records with an invalid timestamp
  26. //TBD link to Working with Out of Order timeseries concept doc
  27. IMPORTANT: For each job, data can only be accepted from a single connection at
  28. a time. It is not currently possible to post data to multiple jobs using wildcards
  29. or a comma-separated list.
  30. ==== Path Parameters
  31. `job_id` (required)::
  32. (string) Identifier for the job
  33. ==== Query Parameters
  34. `reset_start`::
  35. (string) Specifies the start of the bucket resetting range
  36. `reset_end`::
  37. (string) Specifies the end of the bucket resetting range
  38. ==== Request Body
  39. A sequence of one or more JSON documents containing the data to be analyzed.
  40. Only whitespace characters are permitted in between the documents.
  41. ==== Authorization
  42. You must have `manage_ml`, or `manage` cluster privileges to use this API.
  43. For more information, see
  44. {xpack-ref}/security-privileges.html[Security Privileges].
  45. //<<privileges-list-cluster>>.
  46. ==== Examples
  47. The following example posts data from the it_ops_new_kpi.json file to the `it_ops_new_kpi` job:
  48. [source,js]
  49. --------------------------------------------------
  50. $ curl -s -H "Content-type: application/json"
  51. -X POST http:\/\/localhost:9200/_xpack/ml/anomaly_detectors/it_ops_new_kpi/_data
  52. --data-binary @it_ops_new_kpi.json
  53. --------------------------------------------------
  54. When the data is sent, you receive information about the operational progress of the job.
  55. For example:
  56. [source,js]
  57. ----
  58. {
  59. "job_id":"it_ops_new_kpi",
  60. "processed_record_count":21435,
  61. "processed_field_count":64305,
  62. "input_bytes":2589063,
  63. "input_field_count":85740,
  64. "invalid_date_count":0,
  65. "missing_field_count":0,
  66. "out_of_order_timestamp_count":0,
  67. "empty_bucket_count":16,
  68. "sparse_bucket_count":0,
  69. "bucket_count":2165,
  70. "earliest_record_timestamp":1454020569000,
  71. "latest_record_timestamp":1455318669000,
  72. "last_data_time":1491952300658,
  73. "latest_empty_bucket_timestamp":1454541600000,
  74. "input_record_count":21435
  75. }
  76. ----
  77. For more information about these properties, see <<ml-jobstats,Job Stats>>.