downsample-data-stream.asciidoc 3.3 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126
  1. [role="xpack"]
  2. [[indices-downsample-data-stream]]
  3. === Downsample index API
  4. ++++
  5. <titleabbrev>Downsample</titleabbrev>
  6. ++++
  7. .New API reference
  8. [sidebar]
  9. --
  10. For the most up-to-date API details, refer to {api-es}/group/endpoint-data-stream[Data stream APIs].
  11. --
  12. Aggregates a time series (TSDS) index and stores
  13. pre-computed statistical summaries (`min`, `max`, `sum`, `value_count` and
  14. `avg`) for each metric field grouped by a configured time interval. For example,
  15. a TSDS index that contains metrics sampled every 10 seconds can be downsampled
  16. to an hourly index. All documents within an hour interval are summarized and
  17. stored as a single document in the downsample index.
  18. // tag::downsample-example[]
  19. ////
  20. [source,console]
  21. ----
  22. PUT /my-time-series-index
  23. {
  24. "settings": {
  25. "index": {
  26. "mode": "time_series",
  27. "time_series": {
  28. "start_time": "2022-06-10T00:00:00Z",
  29. "end_time": "2022-06-30T23:59:59Z"
  30. },
  31. "routing_path": [
  32. "test.namespace"
  33. ],
  34. "number_of_replicas": 0,
  35. "number_of_shards": 2
  36. }
  37. },
  38. "mappings": {
  39. "properties": {
  40. "@timestamp": {
  41. "type": "date"
  42. },
  43. "metric": {
  44. "type": "long",
  45. "time_series_metric": "gauge"
  46. },
  47. "dimension": {
  48. "type": "keyword",
  49. "time_series_dimension": true
  50. }
  51. }
  52. }
  53. }
  54. PUT /my-time-series-index/_block/write
  55. ----
  56. // TEST
  57. ////
  58. [source,console]
  59. ----
  60. POST /my-time-series-index/_downsample/my-downsampled-time-series-index
  61. {
  62. "fixed_interval": "1d"
  63. }
  64. ----
  65. // TEST[continued]
  66. ////
  67. [source,console]
  68. ----
  69. DELETE /my-time-series-index*
  70. DELETE _data_stream/*
  71. DELETE _index_template/*
  72. ----
  73. // TEST[continued]
  74. ////
  75. // end::downsample-example[]
  76. Check the <<downsampling,Downsampling>> documentation for an overview, details about the downsampling process, and examples of running downsampling manually and as part of an ILM policy.
  77. [[downsample-api-request]]
  78. ==== {api-request-title}
  79. `POST /<source-index>/_downsample/<output-downsampled-index>`
  80. [[downsample-api-prereqs]]
  81. ==== {api-prereq-title}
  82. * Only indices in a <<tsds,time series data stream>> are supported.
  83. * If the {es} {security-features} are enabled, you must have the `all`
  84. or `manage` <<privileges-list-indices,index privilege>> for the data stream.
  85. * Neither <<field-and-document-access-control,field nor document level security>> can be defined on the source index.
  86. * The source index must be read only (`index.blocks.write: true`).
  87. [[downsample-api-path-params]]
  88. ==== {api-path-parms-title}
  89. `<source-index>`::
  90. (Optional, string) Name of the time series index to downsample.
  91. `<output-downsampled_index>`::
  92. +
  93. --
  94. (Required, string) Name of the index to create.
  95. include::{es-ref-dir}/indices/create-index.asciidoc[tag=index-name-reqs]
  96. --
  97. [role="child_attributes"]
  98. [[downsample-api-query-parms]]
  99. ==== {api-query-parms-title}
  100. `fixed_interval`:: (Required, <<time-units,time units>>) The interval at which
  101. to aggregate the original time series index. For example, `60m` produces a
  102. document for each 60 minute (hourly) interval. This follows standard time
  103. formatting syntax as used elsewhere in {es}.
  104. +
  105. NOTE: Smaller, more granular intervals take up proportionally more space.