rollup-caps.asciidoc 5.2 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186
  1. [role="xpack"]
  2. [testenv="basic"]
  3. [[rollup-get-rollup-caps]]
  4. === Get rollup job capabilities API
  5. ++++
  6. <titleabbrev>Get rollup caps</titleabbrev>
  7. ++++
  8. experimental[]
  9. This API returns the capabilities of any rollup jobs that have been configured
  10. for a specific index or index pattern.
  11. This API is useful because a rollup job is often configured to rollup only a
  12. subset of fields from the source index. Furthermore, only certain aggregations
  13. can be configured for various fields, leading to a limited subset of
  14. functionality depending on that configuration.
  15. This API enables you to inspect an index and determine:
  16. 1. Does this index have associated rollup data somewhere in the cluster?
  17. 2. If yes to the first question, what fields were rolled up, what aggregations
  18. can be performed, and where does the data live?
  19. ==== Request
  20. `GET _rollup/data/{index}`
  21. //===== Description
  22. ==== Path Parameters
  23. `index`::
  24. (string) Index, indices or index-pattern to return rollup capabilities for. `_all` may be used to fetch
  25. rollup capabilities from all jobs
  26. ==== Request Body
  27. There is no request body for the Get Rollup Caps API.
  28. ==== Authorization
  29. You must have `monitor`, `monitor_rollup`, `manage` or `manage_rollup` cluster privileges to use this API.
  30. For more information, see
  31. {xpack-ref}/security-privileges.html[Security Privileges].
  32. ==== Examples
  33. Imagine we have an index named `sensor-1` full of raw data. We know that the data will grow over time, so there
  34. will be a `sensor-2`, `sensor-3`, etc. Let's create a Rollup job that targets the index pattern `sensor-*` to accommodate
  35. this future scaling:
  36. [source,js]
  37. --------------------------------------------------
  38. PUT _rollup/job/sensor
  39. {
  40. "index_pattern": "sensor-*",
  41. "rollup_index": "sensor_rollup",
  42. "cron": "*/30 * * * * ?",
  43. "page_size" :1000,
  44. "groups" : {
  45. "date_histogram": {
  46. "field": "timestamp",
  47. "fixed_interval": "1h",
  48. "delay": "7d"
  49. },
  50. "terms": {
  51. "fields": ["node"]
  52. }
  53. },
  54. "metrics": [
  55. {
  56. "field": "temperature",
  57. "metrics": ["min", "max", "sum"]
  58. },
  59. {
  60. "field": "voltage",
  61. "metrics": ["avg"]
  62. }
  63. ]
  64. }
  65. --------------------------------------------------
  66. // CONSOLE
  67. // TEST[setup:sensor_index]
  68. We can then retrieve the rollup capabilities of that index pattern (`sensor-*`) via the following command:
  69. [source,js]
  70. --------------------------------------------------
  71. GET _rollup/data/sensor-*
  72. --------------------------------------------------
  73. // CONSOLE
  74. // TEST[continued]
  75. Which will yield the following response:
  76. [source,js]
  77. ----
  78. {
  79. "sensor-*" : {
  80. "rollup_jobs" : [
  81. {
  82. "job_id" : "sensor",
  83. "rollup_index" : "sensor_rollup",
  84. "index_pattern" : "sensor-*",
  85. "fields" : {
  86. "node" : [
  87. {
  88. "agg" : "terms"
  89. }
  90. ],
  91. "temperature" : [
  92. {
  93. "agg" : "min"
  94. },
  95. {
  96. "agg" : "max"
  97. },
  98. {
  99. "agg" : "sum"
  100. }
  101. ],
  102. "timestamp" : [
  103. {
  104. "agg" : "date_histogram",
  105. "time_zone" : "UTC",
  106. "fixed_interval" : "1h",
  107. "delay": "7d"
  108. }
  109. ],
  110. "voltage" : [
  111. {
  112. "agg" : "avg"
  113. }
  114. ]
  115. }
  116. }
  117. ]
  118. }
  119. }
  120. ----
  121. // TESTRESPONSE
  122. The response that is returned contains information that is similar to the original Rollup configuration, but formatted
  123. differently. First, there are some house-keeping details: the Rollup job's ID, the index that holds the rolled data,
  124. the index pattern that the job was targeting.
  125. Next it shows a list of fields that contain data eligible for rollup searches. Here we see four fields: `node`, `temperature`,
  126. `timestamp` and `voltage`. Each of these fields list the aggregations that are possible. For example, you can use a min, max
  127. or sum aggregation on the `temperature` field, but only a `date_histogram` on `timestamp`.
  128. Note that the `rollup_jobs` element is an array; there can be multiple, independent jobs configured for a single index
  129. or index pattern. Each of these jobs may have different configurations, so the API returns a list of all the various
  130. configurations available.
  131. We could also retrieve the same information with a request to `_all`:
  132. [source,js]
  133. --------------------------------------------------
  134. GET _rollup/data/_all
  135. --------------------------------------------------
  136. // CONSOLE
  137. // TEST[continued]
  138. But note that if we use the concrete index name (`sensor-1`), we'll retrieve no rollup capabilities:
  139. [source,js]
  140. --------------------------------------------------
  141. GET _rollup/data/sensor-1
  142. --------------------------------------------------
  143. // CONSOLE
  144. // TEST[continued]
  145. [source,js]
  146. ----
  147. {
  148. }
  149. ----
  150. // TESTRESPONSE
  151. Why is this? The original rollup job was configured against a specific index pattern (`sensor-*`) not a concrete index
  152. (`sensor-1`). So while the index belongs to the pattern, the rollup job is only valid across the entirety of the pattern
  153. not just one of it's containing indices. So for that reason, the Rollup Capabilities API only returns information based
  154. on the originally configured index name or pattern.