|
|
@@ -2,14 +2,255 @@
|
|
|
=== Using {esql} in {kib}
|
|
|
|
|
|
++++
|
|
|
-<titleabbrev>Kibana</titleabbrev>
|
|
|
+<titleabbrev>Using {esql} in {kib}</titleabbrev>
|
|
|
++++
|
|
|
|
|
|
+You can use {esql} in {kib} to query and aggregate your data, create
|
|
|
+visualizations, and set up alerts.
|
|
|
|
|
|
-Use {esql} in Discover to explore a data set. From the data view dropdown,
|
|
|
-select *Try {esql}* to get started.
|
|
|
+This guide shows you how to use {esql} in Kibana. To follow along with the
|
|
|
+queries, load the "Sample web logs" sample data set by clicking *Try sample
|
|
|
+data* from the {kib} Home, selecting *Other sample data sets*, and clicking *Add
|
|
|
+data* on the *Sample web logs* card.
|
|
|
|
|
|
-NOTE: {esql} queries in Discover and Lens are subject to the time range selected
|
|
|
-with the time filter.
|
|
|
+[discrete]
|
|
|
+[[esql-kibana-get-started]]
|
|
|
+=== Get started with {esql}
|
|
|
|
|
|
+To get started with {esql} in Discover, open the main menu and select
|
|
|
+*Discover*. Next, from the Data views menu, select *Try ES|QL*.
|
|
|
|
|
|
+image::images/esql/esql-data-view-menu.png[align="center",width=33%]
|
|
|
+
|
|
|
+The ability to select {esql} from the Data views menu can be enabled and
|
|
|
+disabled using the `discover:enableESQL` setting from
|
|
|
+{kibana-ref}/advanced-options.html[Advanced Settings].
|
|
|
+
|
|
|
+[discrete]
|
|
|
+[[esql-kibana-query-bar]]
|
|
|
+=== The query bar
|
|
|
+
|
|
|
+After switching to {esql} mode, the query bar shows a sample query. For example:
|
|
|
+
|
|
|
+[source,esql]
|
|
|
+----
|
|
|
+from kibana_sample_data_logs | limit 10
|
|
|
+----
|
|
|
+
|
|
|
+Every query starts with a <<esql-commands,source command>>. In this query, the
|
|
|
+source command is <<esql-from>>. `FROM` retrieves data from data streams, indices, or
|
|
|
+aliases. In this example, the data is retrieved from `kibana_sample_data_logs`.
|
|
|
+
|
|
|
+A source command can be followed by one or more <<esql-commands,processing
|
|
|
+commands>>. In this query, the processing command is <<esql-limit>>. `LIMIT`
|
|
|
+limits the number of rows that are retrieved.
|
|
|
+
|
|
|
+TIP: Click the help icon (image:images/esql/esql-icon-help.svg[]) to open the
|
|
|
+in-product reference documentation for all commands and functions.
|
|
|
+
|
|
|
+To make it easier to write queries, auto-complete offers suggestions with
|
|
|
+possible commands and functions:
|
|
|
+
|
|
|
+image::images/esql/esql-kibana-auto-complete.png[align="center"]
|
|
|
+
|
|
|
+[NOTE]
|
|
|
+====
|
|
|
+{esql} keywords are case-insensitive. The following query is identical to the
|
|
|
+previous one:
|
|
|
+
|
|
|
+[source,esql]
|
|
|
+----
|
|
|
+FROM kibana_sample_data_logs | LIMIT 10
|
|
|
+----
|
|
|
+====
|
|
|
+
|
|
|
+[discrete]
|
|
|
+==== Expand the query bar
|
|
|
+
|
|
|
+For readability, you can put each processing command on a new line. The
|
|
|
+following query is identical to the previous one:
|
|
|
+
|
|
|
+[source,esql]
|
|
|
+----
|
|
|
+FROM kibana_sample_data_logs
|
|
|
+| LIMIT 10
|
|
|
+----
|
|
|
+
|
|
|
+To make it easier to write multi-line queries, click the double-headed arrow
|
|
|
+button (image:images/esql/esql-icon-expand-query-bar.svg[]) to expand the query
|
|
|
+bar:
|
|
|
+
|
|
|
+image::images/esql/esql-expanded-query-bar.png[align="center"]
|
|
|
+
|
|
|
+To return to a compact query bar, click the minimize editor button
|
|
|
+(image:images/esql/esql-icon-minimize-query-bar.svg[]).
|
|
|
+
|
|
|
+[discrete]
|
|
|
+==== Warnings
|
|
|
+
|
|
|
+A query may result in warnings, for example when querying an unsupported field
|
|
|
+type. When that happens, a warning symbol is shown in the query bar. To see the
|
|
|
+detailed warning, expand the query bar, and click *warnings*.
|
|
|
+
|
|
|
+[discrete]
|
|
|
+[[esql-kibana-results-table]]
|
|
|
+=== The results table
|
|
|
+
|
|
|
+For the example query, the results table shows 10 rows. Omitting the `LIMIT`
|
|
|
+command, the results table defaults to up to 500 rows. Using `LIMIT`, you can
|
|
|
+increase the limit to up to 10,000 rows.
|
|
|
+
|
|
|
+NOTE: the 10,000 row limit only applies to the number of rows that are retrieved
|
|
|
+by the query and displayed in Discover. Any query or aggregation runs on the
|
|
|
+full data set.
|
|
|
+
|
|
|
+Each row shows two columns for the example query: a column with the `@timestamp`
|
|
|
+field and a column with the full document. To display specific fields from the
|
|
|
+documents, use the <<esql-keep>> command:
|
|
|
+
|
|
|
+[source,esql]
|
|
|
+----
|
|
|
+FROM kibana_sample_data_logs
|
|
|
+| KEEP @timestamp, bytes, geo.dest
|
|
|
+----
|
|
|
+
|
|
|
+To display all fields as separate columns, use `KEEP *`:
|
|
|
+
|
|
|
+[source,esql]
|
|
|
+----
|
|
|
+FROM kibana_sample_data_logs
|
|
|
+| KEEP *
|
|
|
+----
|
|
|
+
|
|
|
+NOTE: The maximum number of columns in Discover is 50. If a query returns more
|
|
|
+than 50 columns, Discover only shows the first 50.
|
|
|
+
|
|
|
+[discrete]
|
|
|
+==== Sorting
|
|
|
+
|
|
|
+To sort on one of the columns, click the column name you want to sort on and
|
|
|
+select the sort order. Note that this performs client-side sorting. It only
|
|
|
+sorts the rows that were retrieved by the query, which may not be the full
|
|
|
+dataset because of the (implicit) limit. To sort the full data set, use the
|
|
|
+<<esql-sort>> command:
|
|
|
+
|
|
|
+[source,esql]
|
|
|
+----
|
|
|
+FROM kibana_sample_data_logs
|
|
|
+| KEEP @timestamp, bytes, geo.dest
|
|
|
+| SORT bytes DESC
|
|
|
+----
|
|
|
+
|
|
|
+[discrete]
|
|
|
+[[esql-kibana-time-filter]]
|
|
|
+=== Time filtering
|
|
|
+
|
|
|
+To display data within a specified time range, use the
|
|
|
+{kibana-ref}/set-time-filter.html[time filter]. The time filter is only enabled
|
|
|
+when the indices you're querying have a field called `@timestamp`.
|
|
|
+
|
|
|
+If your indices do not have a timestamp field called `@timestamp`, you can limit
|
|
|
+the time range using the <<esql-where>> command and the <<esql-now>> function.
|
|
|
+For example, if the timestamp field is called `timestamp`, to query the last 15
|
|
|
+minutes of data:
|
|
|
+[source,esql]
|
|
|
+----
|
|
|
+FROM kibana_sample_data_logs
|
|
|
+| WHERE timestamp > NOW() - 15minutes
|
|
|
+----
|
|
|
+
|
|
|
+[discrete]
|
|
|
+[[esql-kibana-visualizations]]
|
|
|
+=== Analyze and visualize data
|
|
|
+
|
|
|
+Between the query bar and the results table, Discover shows a date histogram
|
|
|
+visualization. If the indices you're querying do not contain an `@timestamp`
|
|
|
+field, the histogram is not shown.
|
|
|
+
|
|
|
+The visualization adapts to the query. A query's nature determines the type of
|
|
|
+visualization. For example, this query aggregates the total number of bytes per
|
|
|
+destination country:
|
|
|
+
|
|
|
+[source,esql]
|
|
|
+----
|
|
|
+FROM kibana_sample_data_logs
|
|
|
+| STATS total_bytes = SUM(bytes) BY geo.dest
|
|
|
+| SORT total_bytes DESC
|
|
|
+| LIMIT 3
|
|
|
+----
|
|
|
+
|
|
|
+The resulting visualization is a bar chart showing the top 3 countries:
|
|
|
+
|
|
|
+image::images/esql/esql-kibana-bar-chart.png[align="center"]
|
|
|
+
|
|
|
+To change the visualization into another type, click the visualization type
|
|
|
+dropdown:
|
|
|
+
|
|
|
+image::images/esql/esql-kibana-visualization-type.png[align="center",width=33%]
|
|
|
+
|
|
|
+To make other changes to the visualization, like the axes and colors, click the
|
|
|
+pencil button (image:images/esql/esql-icon-edit-visualization.svg[]). This opens
|
|
|
+an in-line editor:
|
|
|
+
|
|
|
+image::images/esql/esql-kibana-in-line-editor.png[align="center"]
|
|
|
+
|
|
|
+You can save the visualization to a new or existing dashboard by clicking the
|
|
|
+save button (image:images/esql/esql-icon-save-visualization.svg[]). Once saved
|
|
|
+to a dashboard, you can continue to make changes to visualization. Click the
|
|
|
+options button in the top-right (image:images/esql/esql-icon-options.svg[]) and
|
|
|
+select *Edit ESQL visualization* to open the in-line editor:
|
|
|
+
|
|
|
+image::images/esql/esql-kibana-edit-on-dashboard.png[align="center"]
|
|
|
+
|
|
|
+[discrete]
|
|
|
+[[esql-kibana-enrich]]
|
|
|
+=== Create an enrich policy
|
|
|
+
|
|
|
+The {esql} <<esql-enrich>> command enables you to <<esql-enrich-data,enrich>>
|
|
|
+your query dataset with fields from another dataset. Before you can use
|
|
|
+`ENRICH`, you need to <<esql-set-up-enrich-policy,create and execute an enrich
|
|
|
+policy>>. If a policy exists, it will be suggested by auto-complete. If not,
|
|
|
+click *Click to create* to create one.
|
|
|
+
|
|
|
+image::images/esql/esql-kibana-enrich-autocomplete.png[align="center"]
|
|
|
+
|
|
|
+Next, you can enter a policy name, the policy type, source indices, and
|
|
|
+optionally a query:
|
|
|
+
|
|
|
+image::images/esql/esql-kibana-enrich-step-1.png[align="center",width="50%"]
|
|
|
+
|
|
|
+Click *Next* to select the match field and enrich fields:
|
|
|
+
|
|
|
+image::images/esql/esql-kibana-enrich-step-2.png[align="center",width="50%"]
|
|
|
+
|
|
|
+Finally, click *Create and execute*.
|
|
|
+
|
|
|
+Now, you can use the enrich policy in an {esql} query:
|
|
|
+
|
|
|
+image::images/esql/esql-kibana-enriched-data.png[align="center"]
|
|
|
+
|
|
|
+[discrete]
|
|
|
+[[esql-kibana-alerting-rule]]
|
|
|
+=== Create an alerting rule
|
|
|
+
|
|
|
+You can use {esql} queries to create alerts. From Discover, click *Alerts* and
|
|
|
+select *Create search threshold rule*. This opens a panel that enables you to
|
|
|
+create a rule using an {esql} query. Next, you can test the query, add a
|
|
|
+connector, and save the rule.
|
|
|
+
|
|
|
+image::images/esql/esql-kibana-create-rule.png[align="center",width=50%]
|
|
|
+
|
|
|
+[discrete]
|
|
|
+[[esql-kibana-limitations]]
|
|
|
+=== Limitations
|
|
|
+
|
|
|
+* The user interface to filter data is not enabled when Discover is in {esql}
|
|
|
+mode. To filter data, write a query that uses the <<esql-where>> command
|
|
|
+instead.
|
|
|
+* In {esql} mode, clicking a field in the field list in Discover does not show
|
|
|
+quick statistics for that field.
|
|
|
+* Discover shows no more than 10,000 rows. This limit only applies to the number
|
|
|
+of rows that are retrieved by the query and displayed in Discover. Any query or
|
|
|
+aggregation runs on the full data set.
|
|
|
+* Discover shows no more than 50 columns. If a query returns
|
|
|
+more than 50 columns, Discover only shows the first 50.
|
Abdon Pijpelink
