moving-percentiles-aggregation.asciidoc 5.6 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165
  1. [role="xpack"]
  2. [testenv="basic"]
  3. [[search-aggregations-pipeline-moving-percentiles-aggregation]]
  4. === Moving percentiles aggregation
  5. ++++
  6. <titleabbrev>Moving percentiles</titleabbrev>
  7. ++++
  8. Given an ordered series of <<search-aggregations-metrics-percentile-aggregation, percentiles>>, the Moving Percentile aggregation
  9. will slide a window across those percentiles and allow the user to compute the cumulative percentile.
  10. This is conceptually very similar to the <<search-aggregations-pipeline-movfn-aggregation, Moving Function>> pipeline aggregation,
  11. except it works on the percentiles sketches instead of the actual buckets values.
  12. ==== Syntax
  13. A `moving_percentiles` aggregation looks like this in isolation:
  14. [source,js]
  15. --------------------------------------------------
  16. {
  17. "moving_percentiles": {
  18. "buckets_path": "the_percentile",
  19. "window": 10
  20. }
  21. }
  22. --------------------------------------------------
  23. // NOTCONSOLE
  24. [[moving-percentiles-params]]
  25. .`moving_percentiles` Parameters
  26. [options="header"]
  27. |===
  28. |Parameter Name |Description |Required |Default Value
  29. |`buckets_path` |Path to the percentile of interest (see <<buckets-path-syntax, `buckets_path` Syntax>> for more details |Required |
  30. |`window` |The size of window to "slide" across the histogram. |Required |
  31. |`shift` |<<shift-parameter, Shift>> of window position. |Optional | 0
  32. |===
  33. `moving_percentiles` aggregations must be embedded inside of a `histogram` or `date_histogram` aggregation. They can be
  34. embedded like any other metric aggregation:
  35. [source,console]
  36. --------------------------------------------------
  37. POST /_search
  38. {
  39. "size": 0,
  40. "aggs": {
  41. "my_date_histo": { <1>
  42. "date_histogram": {
  43. "field": "date",
  44. "calendar_interval": "1M"
  45. },
  46. "aggs": {
  47. "the_percentile": { <2>
  48. "percentiles": {
  49. "field": "price",
  50. "percents": [ 1.0, 99.0 ]
  51. }
  52. },
  53. "the_movperc": {
  54. "moving_percentiles": {
  55. "buckets_path": "the_percentile", <3>
  56. "window": 10
  57. }
  58. }
  59. }
  60. }
  61. }
  62. }
  63. --------------------------------------------------
  64. // TEST[setup:sales]
  65. <1> A `date_histogram` named "my_date_histo" is constructed on the "timestamp" field, with one-day intervals
  66. <2> A `percentile` metric is used to calculate the percentiles of a field.
  67. <3> Finally, we specify a `moving_percentiles` aggregation which uses "the_percentile" sketch as its input.
  68. Moving percentiles are built by first specifying a `histogram` or `date_histogram` over a field. You then add
  69. a percentile metric inside of that histogram. Finally, the `moving_percentiles` is embedded inside the histogram.
  70. The `buckets_path` parameter is then used to "point" at the percentiles aggregation inside of the histogram (see
  71. <<buckets-path-syntax>> for a description of the syntax for `buckets_path`).
  72. And the following may be the response:
  73. [source,console-result]
  74. --------------------------------------------------
  75. {
  76. "took": 11,
  77. "timed_out": false,
  78. "_shards": ...,
  79. "hits": ...,
  80. "aggregations": {
  81. "my_date_histo": {
  82. "buckets": [
  83. {
  84. "key_as_string": "2015/01/01 00:00:00",
  85. "key": 1420070400000,
  86. "doc_count": 3,
  87. "the_percentile": {
  88. "values": {
  89. "1.0": 150.0,
  90. "99.0": 200.0
  91. }
  92. }
  93. },
  94. {
  95. "key_as_string": "2015/02/01 00:00:00",
  96. "key": 1422748800000,
  97. "doc_count": 2,
  98. "the_percentile": {
  99. "values": {
  100. "1.0": 10.0,
  101. "99.0": 50.0
  102. }
  103. },
  104. "the_movperc": {
  105. "values": {
  106. "1.0": 150.0,
  107. "99.0": 200.0
  108. }
  109. }
  110. },
  111. {
  112. "key_as_string": "2015/03/01 00:00:00",
  113. "key": 1425168000000,
  114. "doc_count": 2,
  115. "the_percentile": {
  116. "values": {
  117. "1.0": 175.0,
  118. "99.0": 200.0
  119. }
  120. },
  121. "the_movperc": {
  122. "values": {
  123. "1.0": 10.0,
  124. "99.0": 200.0
  125. }
  126. }
  127. }
  128. ]
  129. }
  130. }
  131. }
  132. --------------------------------------------------
  133. // TESTRESPONSE[s/"took": 11/"took": $body.took/]
  134. // TESTRESPONSE[s/"_shards": \.\.\./"_shards": $body._shards/]
  135. // TESTRESPONSE[s/"hits": \.\.\./"hits": $body.hits/]
  136. The output format of the `moving_percentiles` aggregation is inherited from the format of the referenced
  137. <<search-aggregations-metrics-percentile-aggregation,`percentiles`>> aggregation.
  138. Moving percentiles pipeline aggregations always run with `skip` gap policy.
  139. [[moving-percentiles-shift-parameter]]
  140. ==== shift parameter
  141. By default (with `shift = 0`), the window that is offered for calculation is the last `n` values excluding the current bucket.
  142. Increasing `shift` by 1 moves starting window position by `1` to the right.
  143. - To include current bucket to the window, use `shift = 1`.
  144. - For center alignment (`n / 2` values before and after the current bucket), use `shift = window / 2`.
  145. - For right alignment (`n` values after the current bucket), use `shift = window`.
  146. If either of window edges moves outside the borders of data series, the window shrinks to include available values only.