Browse Source

ESQL: Document MV_SLICE limitations (#114162) (#114348)

`MV_SLICE` is useful, but loading values from lucene frequently sorts
them so `MV_SLICE` is not as useful as you think it is. It's mostly for
after, say, a `SPLIT`. This documents that and adds a link to the
section on multivalues.

It also moves similar docs to a separate paragraph in the docs for
easier reading.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Nik Everett 1 year ago
parent
commit
d6dfa71576

+ 7 - 1
docs/reference/esql/functions/description/mv_first.asciidoc

@@ -2,4 +2,10 @@
 
 *Description*
 
-Converts a multivalued expression into a single valued column containing the first value. This is most useful when reading from a function that emits multivalued columns in a known order like <<esql-split>>.  The order that <<esql-multivalued-fields, multivalued fields>> are read from underlying storage is not guaranteed. It is *frequently* ascending, but don't rely on that. If you need the minimum value use <<esql-mv_min>> instead of `MV_FIRST`. `MV_MIN` has optimizations for sorted values so there isn't a performance benefit to `MV_FIRST`.
+Converts a multivalued expression into a single valued column containing the first value. This is most useful when reading from a function that emits multivalued columns in a known order like <<esql-split>>.
+
+The order that <<esql-multivalued-fields, multivalued fields>> are read from
+underlying storage is not guaranteed. It is *frequently* ascending, but don't
+rely on that. If you need the minimum value use <<esql-mv_min>> instead of
+`MV_FIRST`. `MV_MIN` has optimizations for sorted values so there isn't a
+performance benefit to `MV_FIRST`.

+ 7 - 1
docs/reference/esql/functions/description/mv_last.asciidoc

@@ -2,4 +2,10 @@
 
 *Description*
 
-Converts a multivalue expression into a single valued column containing the last value. This is most useful when reading from a function that emits multivalued columns in a known order like <<esql-split>>.  The order that <<esql-multivalued-fields, multivalued fields>> are read from underlying storage is not guaranteed. It is *frequently* ascending, but don't rely on that. If you need the maximum value use <<esql-mv_max>> instead of `MV_LAST`. `MV_MAX` has optimizations for sorted values so there isn't a performance benefit to `MV_LAST`.
+Converts a multivalue expression into a single valued column containing the last value. This is most useful when reading from a function that emits multivalued columns in a known order like <<esql-split>>.
+
+The order that <<esql-multivalued-fields, multivalued fields>> are read from
+underlying storage is not guaranteed. It is *frequently* ascending, but don't
+rely on that. If you need the maximum value use <<esql-mv_max>> instead of
+`MV_LAST`. `MV_MAX` has optimizations for sorted values so there isn't a
+performance benefit to `MV_LAST`.

+ 5 - 1
docs/reference/esql/functions/description/mv_slice.asciidoc

@@ -2,4 +2,8 @@
 
 *Description*
 
-Returns a subset of the multivalued field using the start and end index values.
+Returns a subset of the multivalued field using the start and end index values. This is most useful when reading from a function that emits multivalued columns in a known order like <<esql-split>> or <<esql-mv_sort>>.
+
+The order that <<esql-multivalued-fields, multivalued fields>> are read from
+underlying storage is not guaranteed. It is *frequently* ascending, but don't
+rely on that.

+ 1 - 1
docs/reference/esql/functions/kibana/definition/mv_first.json

@@ -2,7 +2,7 @@
   "comment" : "This is generated by ESQL's AbstractFunctionTestCase. Do no edit it. See ../README.md for how to regenerate it.",
   "type" : "eval",
   "name" : "mv_first",
-  "description" : "Converts a multivalued expression into a single valued column containing the\nfirst value. This is most useful when reading from a function that emits\nmultivalued columns in a known order like <<esql-split>>.\n\nThe order that <<esql-multivalued-fields, multivalued fields>> are read from\nunderlying storage is not guaranteed. It is *frequently* ascending, but don't\nrely on that. If you need the minimum value use <<esql-mv_min>> instead of\n`MV_FIRST`. `MV_MIN` has optimizations for sorted values so there isn't a\nperformance benefit to `MV_FIRST`.",
+  "description" : "Converts a multivalued expression into a single valued column containing the\nfirst value. This is most useful when reading from a function that emits\nmultivalued columns in a known order like <<esql-split>>.",
   "signatures" : [
     {
       "params" : [

+ 1 - 1
docs/reference/esql/functions/kibana/definition/mv_last.json

@@ -2,7 +2,7 @@
   "comment" : "This is generated by ESQL's AbstractFunctionTestCase. Do no edit it. See ../README.md for how to regenerate it.",
   "type" : "eval",
   "name" : "mv_last",
-  "description" : "Converts a multivalue expression into a single valued column containing the last\nvalue. This is most useful when reading from a function that emits multivalued\ncolumns in a known order like <<esql-split>>.\n\nThe order that <<esql-multivalued-fields, multivalued fields>> are read from\nunderlying storage is not guaranteed. It is *frequently* ascending, but don't\nrely on that. If you need the maximum value use <<esql-mv_max>> instead of\n`MV_LAST`. `MV_MAX` has optimizations for sorted values so there isn't a\nperformance benefit to `MV_LAST`.",
+  "description" : "Converts a multivalue expression into a single valued column containing the last\nvalue. This is most useful when reading from a function that emits multivalued\ncolumns in a known order like <<esql-split>>.",
   "signatures" : [
     {
       "params" : [

+ 1 - 1
docs/reference/esql/functions/kibana/definition/mv_slice.json

@@ -2,7 +2,7 @@
   "comment" : "This is generated by ESQL's AbstractFunctionTestCase. Do no edit it. See ../README.md for how to regenerate it.",
   "type" : "eval",
   "name" : "mv_slice",
-  "description" : "Returns a subset of the multivalued field using the start and end index values.",
+  "description" : "Returns a subset of the multivalued field using the start and end index values.\nThis is most useful when reading from a function that emits multivalued columns\nin a known order like <<esql-split>> or <<esql-mv_sort>>.",
   "signatures" : [
     {
       "params" : [

+ 0 - 6
docs/reference/esql/functions/kibana/docs/mv_first.md

@@ -7,12 +7,6 @@ Converts a multivalued expression into a single valued column containing the
 first value. This is most useful when reading from a function that emits
 multivalued columns in a known order like <<esql-split>>.
 
-The order that <<esql-multivalued-fields, multivalued fields>> are read from
-underlying storage is not guaranteed. It is *frequently* ascending, but don't
-rely on that. If you need the minimum value use <<esql-mv_min>> instead of
-`MV_FIRST`. `MV_MIN` has optimizations for sorted values so there isn't a
-performance benefit to `MV_FIRST`.
-
 ```
 ROW a="foo;bar;baz"
 | EVAL first_a = MV_FIRST(SPLIT(a, ";"))

+ 0 - 6
docs/reference/esql/functions/kibana/docs/mv_last.md

@@ -7,12 +7,6 @@ Converts a multivalue expression into a single valued column containing the last
 value. This is most useful when reading from a function that emits multivalued
 columns in a known order like <<esql-split>>.
 
-The order that <<esql-multivalued-fields, multivalued fields>> are read from
-underlying storage is not guaranteed. It is *frequently* ascending, but don't
-rely on that. If you need the maximum value use <<esql-mv_max>> instead of
-`MV_LAST`. `MV_MAX` has optimizations for sorted values so there isn't a
-performance benefit to `MV_LAST`.
-
 ```
 ROW a="foo;bar;baz"
 | EVAL last_a = MV_LAST(SPLIT(a, ";"))

+ 2 - 0
docs/reference/esql/functions/kibana/docs/mv_slice.md

@@ -4,6 +4,8 @@ This is generated by ESQL's AbstractFunctionTestCase. Do no edit it. See ../READ
 
 ### MV_SLICE
 Returns a subset of the multivalued field using the start and end index values.
+This is most useful when reading from a function that emits multivalued columns
+in a known order like <<esql-split>> or <<esql-mv_sort>>.
 
 ```
 row a = [1, 2, 2, 3]

+ 8 - 3
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/FunctionInfo.java

@@ -29,13 +29,18 @@ public @interface FunctionInfo {
     boolean preview() default false;
 
     /**
-     * The description of the function rendered in {@code META FUNCTIONS}
-     * and the docs. These should be complete sentences.
+     * The description of the function rendered in the docs and kibana's
+     * json files that drive their IDE-like experience. These should be
+     * complete sentences but can contain asciidoc syntax. It is rendered
+     * as a single paragraph.
      */
     String description() default "";
 
     /**
-     * Detailed descriptions of the function rendered in the docs.
+     * Detailed descriptions of the function rendered in the docs. This is
+     * rendered as a single paragraph following {@link #description()} in
+     * the docs and is <strong>excluded</strong> from Kibana's IDE-like
+     * experience. It can contain asciidoc syntax.
      */
     String detailedDescription() default "";
 

+ 2 - 2
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/scalar/multivalue/MvFirst.java

@@ -59,8 +59,8 @@ public class MvFirst extends AbstractMultivalueFunction {
         description = """
             Converts a multivalued expression into a single valued column containing the
             first value. This is most useful when reading from a function that emits
-            multivalued columns in a known order like <<esql-split>>.
-
+            multivalued columns in a known order like <<esql-split>>.""",
+        detailedDescription = """
             The order that <<esql-multivalued-fields, multivalued fields>> are read from
             underlying storage is not guaranteed. It is *frequently* ascending, but don't
             rely on that. If you need the minimum value use <<esql-mv_min>> instead of

+ 2 - 2
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/scalar/multivalue/MvLast.java

@@ -59,8 +59,8 @@ public class MvLast extends AbstractMultivalueFunction {
         description = """
             Converts a multivalue expression into a single valued column containing the last
             value. This is most useful when reading from a function that emits multivalued
-            columns in a known order like <<esql-split>>.
-
+            columns in a known order like <<esql-split>>.""",
+        detailedDescription = """
             The order that <<esql-multivalued-fields, multivalued fields>> are read from
             underlying storage is not guaranteed. It is *frequently* ascending, but don't
             rely on that. If you need the maximum value use <<esql-mv_max>> instead of

+ 8 - 1
x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/scalar/multivalue/MvSlice.java

@@ -68,7 +68,14 @@ public class MvSlice extends EsqlScalarFunction implements OptionalArgument, Eva
             "long",
             "text",
             "version" },
-        description = "Returns a subset of the multivalued field using the start and end index values.",
+        description = """
+            Returns a subset of the multivalued field using the start and end index values.
+            This is most useful when reading from a function that emits multivalued columns
+            in a known order like <<esql-split>> or <<esql-mv_sort>>.""",
+        detailedDescription = """
+            The order that <<esql-multivalued-fields, multivalued fields>> are read from
+            underlying storage is not guaranteed. It is *frequently* ascending, but don't
+            rely on that.""",
         examples = { @Example(file = "ints", tag = "mv_slice_positive"), @Example(file = "ints", tag = "mv_slice_negative") }
     )
     public MvSlice(