rollup-caps.asciidoc 5.4 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192
  1. [role="xpack"]
  2. [[rollup-get-rollup-caps]]
  3. === Get {rollup-job} capabilities API
  4. ++++
  5. <titleabbrev>Get rollup caps</titleabbrev>
  6. ++++
  7. Returns the capabilities of any {rollup-jobs} that have been configured for a
  8. specific index or index pattern.
  9. experimental[]
  10. NOTE: For version 8.5 and above we recommend <<downsampling,downsampling>> over
  11. rollups as a way to reduce your storage costs for time series data.
  12. [[rollup-get-rollup-caps-request]]
  13. ==== {api-request-title}
  14. `GET _rollup/data/<index>`
  15. [[rollup-get-rollup-caps-prereqs]]
  16. ==== {api-prereq-title}
  17. * If the {es} {security-features} are enabled, you must have `monitor`,
  18. `monitor_rollup`, `manage` or `manage_rollup` cluster privileges to use this API.
  19. For more information, see <<security-privileges>>.
  20. [[rollup-get-rollup-caps-desc]]
  21. ==== {api-description-title}
  22. This API is useful because a {rollup-job} is often configured to rollup only a
  23. subset of fields from the source index. Furthermore, only certain aggregations
  24. can be configured for various fields, leading to a limited subset of
  25. functionality depending on that configuration.
  26. This API enables you to inspect an index and determine:
  27. 1. Does this index have associated rollup data somewhere in the cluster?
  28. 2. If yes to the first question, what fields were rolled up, what aggregations
  29. can be performed, and where does the data live?
  30. [[rollup-get-rollup-path-params]]
  31. ==== {api-path-parms-title}
  32. `<index>`::
  33. (string) Index, indices or index-pattern to return rollup capabilities for.
  34. `_all` may be used to fetch rollup capabilities from all jobs.
  35. [[rollup-get-rollup-example]]
  36. ==== {api-examples-title}
  37. Imagine we have an index named `sensor-1` full of raw data. We know that the
  38. data will grow over time, so there will be a `sensor-2`, `sensor-3`, etc. Let's
  39. create a {rollup-job} that targets the index pattern `sensor-*` to accommodate
  40. this future scaling:
  41. [source,console]
  42. --------------------------------------------------
  43. PUT _rollup/job/sensor
  44. {
  45. "index_pattern": "sensor-*",
  46. "rollup_index": "sensor_rollup",
  47. "cron": "*/30 * * * * ?",
  48. "page_size": 1000,
  49. "groups": {
  50. "date_histogram": {
  51. "field": "timestamp",
  52. "fixed_interval": "1h",
  53. "delay": "7d"
  54. },
  55. "terms": {
  56. "fields": [ "node" ]
  57. }
  58. },
  59. "metrics": [
  60. {
  61. "field": "temperature",
  62. "metrics": [ "min", "max", "sum" ]
  63. },
  64. {
  65. "field": "voltage",
  66. "metrics": [ "avg" ]
  67. }
  68. ]
  69. }
  70. --------------------------------------------------
  71. // TEST[setup:sensor_index]
  72. We can then retrieve the rollup capabilities of that index pattern (`sensor-*`)
  73. via the following command:
  74. [source,console]
  75. --------------------------------------------------
  76. GET _rollup/data/sensor-*
  77. --------------------------------------------------
  78. // TEST[continued]
  79. Which will yield the following response:
  80. [source,console-result]
  81. ----
  82. {
  83. "sensor-*" : {
  84. "rollup_jobs" : [
  85. {
  86. "job_id" : "sensor",
  87. "rollup_index" : "sensor_rollup",
  88. "index_pattern" : "sensor-*",
  89. "fields" : {
  90. "node" : [
  91. {
  92. "agg" : "terms"
  93. }
  94. ],
  95. "temperature" : [
  96. {
  97. "agg" : "min"
  98. },
  99. {
  100. "agg" : "max"
  101. },
  102. {
  103. "agg" : "sum"
  104. }
  105. ],
  106. "timestamp" : [
  107. {
  108. "agg" : "date_histogram",
  109. "time_zone" : "UTC",
  110. "fixed_interval" : "1h",
  111. "delay": "7d"
  112. }
  113. ],
  114. "voltage" : [
  115. {
  116. "agg" : "avg"
  117. }
  118. ]
  119. }
  120. }
  121. ]
  122. }
  123. }
  124. ----
  125. The response that is returned contains information that is similar to the
  126. original rollup configuration, but formatted differently. First, there are some
  127. house-keeping details: the {rollup-job} ID, the index that holds the rolled data,
  128. and the index pattern that the job was targeting.
  129. Next it shows a list of fields that contain data eligible for rollup searches.
  130. Here we see four fields: `node`, `temperature`, `timestamp` and `voltage`. Each
  131. of these fields list the aggregations that are possible. For example, you can
  132. use a min, max or sum aggregation on the `temperature` field, but only a
  133. `date_histogram` on `timestamp`.
  134. Note that the `rollup_jobs` element is an array; there can be multiple,
  135. independent jobs configured for a single index or index pattern. Each of these
  136. jobs may have different configurations, so the API returns a list of all the
  137. various configurations available.
  138. We could also retrieve the same information with a request to `_all`:
  139. [source,console]
  140. --------------------------------------------------
  141. GET _rollup/data/_all
  142. --------------------------------------------------
  143. // TEST[continued]
  144. But note that if we use the concrete index name (`sensor-1`), we'll retrieve no
  145. rollup capabilities:
  146. [source,console]
  147. --------------------------------------------------
  148. GET _rollup/data/sensor-1
  149. --------------------------------------------------
  150. // TEST[continued]
  151. [source,console-result]
  152. ----
  153. {
  154. }
  155. ----
  156. Why is this? The original {rollup-job} was configured against a specific index
  157. pattern (`sensor-*`) not a concrete index (`sensor-1`). So while the index
  158. belongs to the pattern, the {rollup-job} is only valid across the entirety of
  159. the pattern not just one of it's containing indices. So for that reason, the
  160. get rollup capabilities API only returns information based on the originally
  161. configured index name or pattern.