|
@@ -26,298 +26,14 @@ ESQL supports these processing commands:
|
|
|
* <<esql-stats-by>>
|
|
|
* <<esql-where>>
|
|
|
|
|
|
-
|
|
|
-[[esql-dissect]]
|
|
|
-=== `DISSECT`
|
|
|
-
|
|
|
-`DISSECT` enables you to extract structured data out of a string. `DISSECT`
|
|
|
-matches the string against a delimiter-based pattern, and extracts the specified
|
|
|
-keys as columns.
|
|
|
-
|
|
|
-Refer to the <<dissect-processor,dissect processor documentation>> for the
|
|
|
-syntax of dissect patterns.
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-ROW a = "1953-01-23T12:15:00Z - some text - 127.0.0.1"
|
|
|
-| DISSECT a "%{Y}-%{M}-%{D}T%{h}:%{m}:%{s}Z - %{msg} - %{ip}"
|
|
|
-----
|
|
|
-
|
|
|
-[[esql-drop]]
|
|
|
-=== `DROP`
|
|
|
-
|
|
|
-Use `DROP` to remove columns from a table:
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| DROP height
|
|
|
-----
|
|
|
-
|
|
|
-Rather than specify each column by name, you can use wildcards to drop all
|
|
|
-columns with a name that matches a pattern:
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| DROP height*
|
|
|
-----
|
|
|
-
|
|
|
-[[esql-eval]]
|
|
|
-=== `EVAL`
|
|
|
-`EVAL` enables you to add new columns to the end of a table:
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| PROJECT first_name, last_name, height
|
|
|
-| EVAL height_feet = height * 3.281, height_cm = height * 100
|
|
|
-----
|
|
|
-
|
|
|
-If the specified column already exists, the existing column will be dropped, and
|
|
|
-the new column will be appended to the table:
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| PROJECT first_name, last_name, height
|
|
|
-| EVAL height = height * 3.281
|
|
|
-----
|
|
|
-
|
|
|
-[discrete]
|
|
|
-==== Functions
|
|
|
-`EVAL` supports various functions for calculating values. Refer to
|
|
|
-<<esql-functions,Functions>> for more information.
|
|
|
-
|
|
|
-[[esql-grok]]
|
|
|
-=== `GROK`
|
|
|
-
|
|
|
-`GROK` enables you to extract structured data out of a string. `GROK` matches
|
|
|
-the string against patterns, based on regular expressions, and extracts the
|
|
|
-specified patterns as columns.
|
|
|
-
|
|
|
-Refer to the <<grok-processor,grok processor documentation>> for the syntax for
|
|
|
-of grok patterns.
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-ROW a = "12 15.5 15.6 true"
|
|
|
-| GROK a "%{NUMBER:b:int} %{NUMBER:c:float} %{NUMBER:d:double} %{WORD:e:boolean}"
|
|
|
-----
|
|
|
-
|
|
|
-[[esql-limit]]
|
|
|
-=== `LIMIT`
|
|
|
-
|
|
|
-The `LIMIT` processing command enables you to limit the number of rows:
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| LIMIT 5
|
|
|
-----
|
|
|
-
|
|
|
-[[esql-mv_expand]]
|
|
|
-=== `MV_EXPAND`
|
|
|
-
|
|
|
-The `MV_EXPAND` processing command expands multivalued fields into one row per value, duplicating other fields:
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-include::{esql-specs}/mv_expand.csv-spec[tag=simple]
|
|
|
-----
|
|
|
-
|
|
|
-[%header,format=dsv,separator=|]
|
|
|
-|===
|
|
|
-include::{esql-specs}/mv_expand.csv-spec[tag=simple-result]
|
|
|
-|===
|
|
|
-
|
|
|
-[[esql-project]]
|
|
|
-=== `PROJECT`
|
|
|
-
|
|
|
-The `PROJECT` command enables you to specify what columns are returned and the
|
|
|
-order in which they are returned.
|
|
|
-
|
|
|
-To limit the columns that are returned, use a comma-separated list of column
|
|
|
-names. The columns are returned in the specified order:
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| PROJECT first_name, last_name, height
|
|
|
-----
|
|
|
-
|
|
|
-Rather than specify each column by name, you can use wildcards to return all
|
|
|
-columns with a name that matches a pattern:
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| PROJECT h*
|
|
|
-----
|
|
|
-
|
|
|
-The asterisk wildcard (`*`) by itself translates to all columns that do not
|
|
|
-match the other arguments. This query will first return all columns with a name
|
|
|
-that starts with an h, followed by all other columns:
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| PROJECT h*, *
|
|
|
-----
|
|
|
-
|
|
|
-[[esql-rename]]
|
|
|
-=== `RENAME`
|
|
|
-
|
|
|
-Use `RENAME` to rename a column. If a column with the new name already exists,
|
|
|
-it will be replaced by the new column.
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| PROJECT first_name, last_name, still_hired
|
|
|
-| RENAME employed = still_hired
|
|
|
-----
|
|
|
-
|
|
|
-Multiple columns can be renamed with a single `RENAME` command:
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| PROJECT first_name, last_name
|
|
|
-| RENAME fn = first_name, ln = last_name
|
|
|
-----
|
|
|
-
|
|
|
-[[esql-sort]]
|
|
|
-=== `SORT`
|
|
|
-Use the `SORT` command to sort rows on one or more fields:
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| PROJECT first_name, last_name, height
|
|
|
-| SORT height
|
|
|
-----
|
|
|
-
|
|
|
-The default sort order is ascending. Set an explicit sort order using `ASC` or
|
|
|
-`DESC`:
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| PROJECT first_name, last_name, height
|
|
|
-| SORT height DESC
|
|
|
-----
|
|
|
-
|
|
|
-If two rows have the same sort key, the original order will be preserved. You
|
|
|
-can provide additional sort expressions to act as tie breakers:
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| PROJECT first_name, last_name, height
|
|
|
-| SORT height DESC, first_name ASC
|
|
|
-----
|
|
|
-
|
|
|
-[discrete]
|
|
|
-==== `null` values
|
|
|
-By default, `null` values are treated as being larger than any other value. With
|
|
|
-an ascending sort order, `null` values are sorted last, and with a descending
|
|
|
-sort order, `null` values are sorted first. You can change that by providing
|
|
|
-`NULLS FIRST` or `NULLS LAST`:
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| PROJECT first_name, last_name, height
|
|
|
-| SORT first_name ASC NULLS FIRST
|
|
|
-----
|
|
|
-
|
|
|
-[[esql-stats-by]]
|
|
|
-=== `STATS ... BY`
|
|
|
-Use `STATS ... BY` to group rows according to a common value and calculate one
|
|
|
-or more aggregated values over the grouped rows.
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| STATS count = COUNT(languages) BY languages
|
|
|
-----
|
|
|
-
|
|
|
-If `BY` is omitted, the output table contains exactly one row with the
|
|
|
-aggregations applied over the entire dataset:
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| STATS avg_lang = AVG(languages)
|
|
|
-----
|
|
|
-
|
|
|
-It's possible to calculate multiple values:
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| STATS avg_lang = AVG(languages), max_lang = MAX(languages)
|
|
|
-----
|
|
|
-
|
|
|
-It's also possible to group by multiple values (only supported for long and
|
|
|
-keyword family fields):
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| EVAL hired = DATE_FORMAT(hire_date, "YYYY")
|
|
|
-| STATS avg_salary = AVG(salary) BY hired, languages.long
|
|
|
-| EVAL avg_salary = ROUND(avg_salary)
|
|
|
-| SORT hired, languages.long
|
|
|
-----
|
|
|
-
|
|
|
-The following aggregation functions are supported:
|
|
|
-
|
|
|
-* `AVG`
|
|
|
-* `COUNT`
|
|
|
-* `COUNT_DISTINCT`
|
|
|
-* `MAX`
|
|
|
-* `MEDIAN`
|
|
|
-* `MEDIAN_ABSOLUTE_DEVIATION`
|
|
|
-* `MIN`
|
|
|
-* `SUM`
|
|
|
-
|
|
|
-[[esql-where]]
|
|
|
-=== `WHERE`
|
|
|
-
|
|
|
-Use `WHERE` to produce a table that contains all the rows from the input table
|
|
|
-for which the provided condition evaluates to `true`:
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| PROJECT first_name, last_name, still_hired
|
|
|
-| WHERE still_hired == true
|
|
|
-----
|
|
|
-
|
|
|
-Which, if `still_hired` is a boolean field, can be simplified to:
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| PROJECT first_name, last_name, still_hired
|
|
|
-| WHERE still_hired
|
|
|
-----
|
|
|
-
|
|
|
-[discrete]
|
|
|
-==== Operators
|
|
|
-
|
|
|
-Refer to <<esql-operators>> for an overview of the supported operators.
|
|
|
-
|
|
|
-[discrete]
|
|
|
-==== Functions
|
|
|
-`WHERE` supports various functions for calculating values. Refer to
|
|
|
-<<esql-functions,Functions>> for more information.
|
|
|
-
|
|
|
-[source,esql]
|
|
|
-----
|
|
|
-FROM employees
|
|
|
-| PROJECT first_name, last_name, height
|
|
|
-| WHERE length(first_name) < 4
|
|
|
-----
|
|
|
+include::processing-commands/dissect.asciidoc[]
|
|
|
+include::processing-commands/drop.asciidoc[]
|
|
|
+include::processing-commands/eval.asciidoc[]
|
|
|
+include::processing-commands/grok.asciidoc[]
|
|
|
+include::processing-commands/limit.asciidoc[]
|
|
|
+include::processing-commands/mv_expand.asciidoc[]
|
|
|
+include::processing-commands/project.asciidoc[]
|
|
|
+include::processing-commands/rename.asciidoc[]
|
|
|
+include::processing-commands/sort.asciidoc[]
|
|
|
+include::processing-commands/stats.asciidoc[]
|
|
|
+include::processing-commands/where.asciidoc[]
|