change-point-aggregation.asciidoc 4.0 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142
  1. [role="xpack"]
  2. [[search-aggregations-change-point-aggregation]]
  3. === Change point aggregation
  4. ++++
  5. <titleabbrev>Change point</titleabbrev>
  6. ++++
  7. experimental::[]
  8. A sibling pipeline that detects, spikes, dips, and change points in a metric. Given a distribution of values
  9. provided by the sibling multi-bucket aggregation, this aggregation indicates the bucket of any spike or dip
  10. and/or the bucket at which the largest change in the distribution of values, if they are statistically significant.
  11. [[change-point-agg-syntax]]
  12. ==== Parameters
  13. `buckets_path`::
  14. (Required, string)
  15. Path to the buckets that contain one set of values in which to detect a change point. There must be at least 22 bucketed
  16. values. Fewer than 1,000 is preferred.
  17. For syntax, see <<buckets-path-syntax>>.
  18. ==== Syntax
  19. A `change_point` aggregation looks like this in isolation:
  20. [source,js]
  21. --------------------------------------------------
  22. {
  23. "change_point": {
  24. "buckets_path": "date_histogram>_count" <1>
  25. }
  26. }
  27. --------------------------------------------------
  28. // NOTCONSOLE
  29. <1> The buckets containing the values to test against.
  30. [[change-point-agg-response]]
  31. ==== Response body
  32. `bucket`::
  33. (Optional, object)
  34. Values of the bucket that indicates the discovered change point. Not returned if no change point was found.
  35. All the aggregations in the bucket are returned as well.
  36. +
  37. .Properties of bucket
  38. [%collapsible%open]
  39. ====
  40. `key`:::
  41. (value)
  42. The key of the bucket matched. Could be string or numeric.
  43. `doc_count`:::
  44. (number)
  45. The document count of the bucket.
  46. ====
  47. `type`::
  48. (object)
  49. The found change point type and its related values. Possible types:
  50. +
  51. --
  52. * `dip`: a significant dip occurs at this change point
  53. * `distribution_change`: the overall distribution of the values has changed significantly
  54. * `non_stationary`: there is no change point, but the values are not from a stationary distribution
  55. * `spike`: a significant spike occurs at this point
  56. * `stationary`: no change point found
  57. * `step_change`: the change indicates a statistically significant step up or down in value distribution
  58. * `trend_change`: there is an overall trend change occurring at this point
  59. --
  60. ==== Example
  61. The following example uses the Kibana sample data logs data set.
  62. [source,js]
  63. --------------------------------------------------
  64. GET kibana_sample_data_logs/_search
  65. {
  66. "aggs": {
  67. "date":{ <1>
  68. "date_histogram": {
  69. "field": "@timestamp",
  70. "fixed_interval": "1d"
  71. },
  72. "aggs": {
  73. "avg": { <2>
  74. "avg": {
  75. "field": "bytes"
  76. }
  77. }
  78. }
  79. },
  80. "change_points_avg": { <3>
  81. "change_point": {
  82. "buckets_path": "date>avg" <4>
  83. }
  84. }
  85. }
  86. }
  87. --------------------------------------------------
  88. // NOTCONSOLE
  89. <1> A date histogram aggregation that creates buckets with one day long
  90. interval.
  91. <2> A sibling aggregation of the `date` aggregation that calculates the average
  92. value of the `bytes` field within every bucket.
  93. <3> The change point detection aggregation configuration object.
  94. <4> The path of the aggregation values to detect change points. In this case,
  95. the input of the change point aggregation is the value of `avg` which is a
  96. sibling aggregation of `date`.
  97. The request returns a response that is similar to the following:
  98. [source,js]
  99. --------------------------------------------------
  100. "change_points_avg" : {
  101. "bucket" : {
  102. "key" : "2023-04-29T00:00:00.000Z", <1>
  103. "doc_count" : 329, <2>
  104. "avg" : { <3>
  105. "value" : 4737.209726443769
  106. }
  107. },
  108. "type" : { <4>
  109. "dip" : {
  110. "p_value" : 3.8999455212466465e-10, <5>
  111. "change_point" : 41 <6>
  112. }
  113. }
  114. }
  115. --------------------------------------------------
  116. // NOTCONSOLE
  117. <1> The bucket key that is the change point.
  118. <2> The number of documents in that bucket.
  119. <3> Aggregated values in the bucket.
  120. <4> Type of change found.
  121. <5> The `p_value` indicates how extreme the change is; lower values indicate greater change.
  122. <6> The specific bucket where the change occurs (indexing starts at `0`).