count-distinct.asciidoc 2.3 KB

12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273
  1. [discrete]
  2. [[esql-agg-count-distinct]]
  3. === `COUNT_DISTINCT`
  4. *Syntax*
  5. [source,esql]
  6. ----
  7. COUNT_DISTINCT(column[, precision_threshold])
  8. ----
  9. *Parameters*
  10. `column`::
  11. Column for which to count the number of distinct values.
  12. `precision_threshold`::
  13. Precision threshold. Refer to <<esql-agg-count-distinct-approximate>>. The
  14. maximum supported value is 40000. Thresholds above this number will have the
  15. same effect as a threshold of 40000. The default value is 3000.
  16. *Description*
  17. Returns the approximate number of distinct values.
  18. [discrete]
  19. [[esql-agg-count-distinct-approximate]]
  20. ==== Counts are approximate
  21. Computing exact counts requires loading values into a set and returning its
  22. size. This doesn't scale when working on high-cardinality sets and/or large
  23. values as the required memory usage and the need to communicate those
  24. per-shard sets between nodes would utilize too many resources of the cluster.
  25. This `COUNT_DISTINCT` function is based on the
  26. https://static.googleusercontent.com/media/research.google.com/fr//pubs/archive/40671.pdf[HyperLogLog++]
  27. algorithm, which counts based on the hashes of the values with some interesting
  28. properties:
  29. include::../../aggregations/metrics/cardinality-aggregation.asciidoc[tag=explanation]
  30. The `COUNT_DISTINCT` function takes an optional second parameter to configure
  31. the precision threshold. The precision_threshold options allows to trade memory
  32. for accuracy, and defines a unique count below which counts are expected to be
  33. close to accurate. Above this value, counts might become a bit more fuzzy. The
  34. maximum supported value is 40000, thresholds above this number will have the
  35. same effect as a threshold of 40000. The default value is `3000`.
  36. *Supported types*
  37. Can take any field type as input.
  38. *Examples*
  39. [source.merge.styled,esql]
  40. ----
  41. include::{esql-specs}/stats_count_distinct.csv-spec[tag=count-distinct]
  42. ----
  43. [%header.monospaced.styled,format=dsv,separator=|]
  44. |===
  45. include::{esql-specs}/stats_count_distinct.csv-spec[tag=count-distinct-result]
  46. |===
  47. With the optional second parameter to configure the precision threshold:
  48. [source.merge.styled,esql]
  49. ----
  50. include::{esql-specs}/stats_count_distinct.csv-spec[tag=count-distinct-precision]
  51. ----
  52. [%header.monospaced.styled,format=dsv,separator=|]
  53. |===
  54. include::{esql-specs}/stats_count_distinct.csv-spec[tag=count-distinct-precision-result]
  55. |===