rollup-api.asciidoc 5.8 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201
  1. [role="xpack"]
  2. [testenv="basic"]
  3. [[rollup-api]]
  4. === Rollup API
  5. ++++
  6. <titleabbrev>Rollup</titleabbrev>
  7. ++++
  8. Aggregates an index's time series data and stores the results in a new read-only
  9. index. For example, you can roll up hourly data into daily or weekly summaries.
  10. [source,console]
  11. ----
  12. POST /my-index-000001/_rollup/rollup-my-index-000001
  13. {
  14. "groups": {
  15. "date_histogram": {
  16. "field": "@timestamp",
  17. "calendar_interval": "1d"
  18. },
  19. "terms": {
  20. "fields": [
  21. "my-keyword-field",
  22. "my-other-keyword-field"
  23. ]
  24. }
  25. },
  26. "metrics": [
  27. {
  28. "field": "my-numeric-field",
  29. "metrics": [
  30. "min",
  31. "max",
  32. "avg"
  33. ]
  34. }
  35. ]
  36. }
  37. ----
  38. // TEST[setup:my_index]
  39. // TEST[s/my-keyword-field/http.request.method/]
  40. // TEST[s/my-other-keyword-field/user.id/]
  41. // TEST[s/my-numeric-field/http.response.bytes/]
  42. [[rollup-api-request]]
  43. ==== {api-request-title}
  44. `PUT /<index>/_rollup/<rollup-index>`
  45. [[rollup-api-prereqs]]
  46. ==== {api-prereq-title}
  47. * If the {es} {security-features} are enabled, you must have the `manage` or
  48. `manage_rollup` cluster privilege to use this API. See <<security-privileges>>.
  49. * You can only roll up an index that contains:
  50. ** A <<date,`date`>> or <<date_nanos,`date_nanos`>> timestamp field
  51. ** One or more <<number,numeric>> fields
  52. [[rollup-api-path-params]]
  53. ==== {api-path-parms-title}
  54. `<index>`::
  55. (Required, string)
  56. Index to roll up. Cannot be a <<data-streams,data stream>> or
  57. <<indices-aliases,index alias>>. Does not support <<multi-index,multi-target
  58. syntax>> or wildcards (`*`).
  59. `<rollup-index>`::
  60. (Required, string)
  61. New index that stores the rollup results. Cannot be an existing index,
  62. a <<data-streams,data stream>>, or an <<indices-aliases,index alias>>.
  63. +
  64. The request creates this index with
  65. <<index-modules-blocks,`index.blocks.write`>> set to `true`. If the source
  66. `<index>` is a backing index for a data stream, this index is a backing index
  67. for the same stream.
  68. [role="child_attributes"]
  69. [[rollup-api-request-body]]
  70. ==== {api-request-body-title}
  71. `groups`::
  72. (Required, object)
  73. Aggregates and stores fields in the rollup.
  74. +
  75. .Properties of `groups`
  76. [%collapsible%open]
  77. =====
  78. `date_histogram`::
  79. (Required,
  80. <<search-aggregations-bucket-datehistogram-aggregation,`date_histogram`
  81. aggregation>> object)
  82. Groups documents based on a provided time interval.
  83. +
  84. .Properties of `date_histogram`
  85. [%collapsible%open]
  86. ======
  87. `field`::
  88. (Required, string)
  89. <<date,`date`>> or <<date_nanos,`date_nanos`>> field containing a timestamp. If
  90. you're rolling up a backing index or using the {ecs-ref}[Elastic Common Schema
  91. (ECS)], we recommend using `@timestamp`.
  92. `calendar_interval` or `fixed_interval`::
  93. (Required, <<time-units,time units>>)
  94. Time interval used to group documents. For differences between
  95. `calendar_interval` and `fixed_interval`, see <<calendar_and_fixed_intervals>>.
  96. +
  97. TIP: Choose this value carefully. You won't be able to use a smaller interval
  98. later. For example, you can't aggregate daily rollups into hourly
  99. summaries. However, smaller time intervals can greatly increase the size of the
  100. resulting rollup index.
  101. `time_zone`::
  102. (Optional, string)
  103. Time zone for the `field`. Valid values are ISO 8601 UTC offsets, such as
  104. `+01:00` or `-08:00`, and IANA time zone IDs, such as `America/Los_Angeles`.
  105. Defaults to `+00:00` (UTC).
  106. ======
  107. `histogram`::
  108. (Optional, <<search-aggregations-bucket-histogram-aggregation,`histogram`
  109. aggregation>> object)
  110. Groups and stores <<number,numeric>> field values based on a provided interval.
  111. +
  112. .Properties of `histogram`
  113. [%collapsible%open]
  114. ======
  115. `fields`::
  116. (Required*, array of strings)
  117. Array of <<number,numeric>> fields to group. If you specify a `histogram`
  118. object, this property is required.
  119. `interval`::
  120. (Required*, integer)
  121. Numeric interval used to group the `fields`. If you specify a `histogram`
  122. object, this property is required.
  123. ======
  124. `terms`::
  125. (Optional, <<search-aggregations-bucket-terms-aggregation,`terms`
  126. aggregation>> object)
  127. Stores values for <<keyword,keyword family>> and <<number,numeric>> fields.
  128. +
  129. .Properties of `terms`
  130. [%collapsible%open]
  131. ======
  132. `fields`::
  133. (Required*, array of strings)
  134. Array of <<keyword,keyword family>> and <<number,numeric>> fields to store. If
  135. you specify a `terms` object, this property is required.
  136. +
  137. TIP: Avoid storing high-cardinality fields. High-cardinality fields can greatly
  138. increase the size of the resulting rollup index.
  139. ======
  140. =====
  141. `metrics`::
  142. (Required, array of objects)
  143. Collects and stores metrics for <<number,numeric>> fields. You must specify at
  144. least one `metrics` object.
  145. +
  146. .Properties of `metrics` objects
  147. [%collapsible%open]
  148. =====
  149. `field`::
  150. (Required, string)
  151. <<number,Numeric>> field to collect metrics for.
  152. `metrics`::
  153. (Required, array of strings)
  154. Array of metrics to collect. Each value corresponds to a
  155. <<search-aggregations-metrics,metric aggregation>>. Valid values are
  156. <<search-aggregations-metrics-min-aggregation,`min`>>,
  157. <<search-aggregations-metrics-max-aggregation,`max`>>,
  158. <<search-aggregations-metrics-sum-aggregation,`sum`>>,
  159. <<search-aggregations-metrics-avg-aggregation,`avg`>>, and
  160. <<search-aggregations-metrics-valuecount-aggregation,`value_count`>>. You must
  161. specify at least one value.
  162. +
  163. NOTE: The rollup index stores these metrics in an
  164. <<aggregate-metric-double,`aggregate_metric_double`>> field. The `avg` metric
  165. stores both the `sum` and `value_count` values. This lets you accurately average
  166. rollups over larger time intervals. For example, you can accurately roll up
  167. hourly averages into daily averages.
  168. =====
  169. `page_size`::
  170. (Optional, integer)
  171. Maximum number of rollup results to process at once. Defaults to `1000`. Larger
  172. values run faster but require more memory.
  173. +
  174. NOTE: This argument only affects the speed and memory usage of the rollup
  175. operation. It does not affect the rollup results.
  176. `timeout`::
  177. (Optional, <<time-units,time value>>)
  178. Time to wait for the request to complete. Defaults to `20s` (20 seconds).