elasticsearch/docs/reference/esql/esql-for-search.asciidoc

[[esql-for-search]]
=== Using {esql} for search

preview::[]

This page provides an overview of how to use {esql} for search use cases.

[TIP]
====
Prefer to get started with a hands-on tutorial? Check out <<esql-search-tutorial>>.
====

The following table summarizes the key search features available in {esql} and when they were introduced.

[cols="1,1,2", options="header"]
|===
|Feature |Available since |Description

|<<esql-search-functions,Full text search functions>>
|8.17
|Perform basic text searches with <<esql-match, match function>> and <<esql-search-operators,match operator (`:`)>>

|<<esql-for-search-query-string,Query string function>>
|8.17
|Execute complex queries with <<esql-qstr,`qstr`>> using Query String syntax

|<<esql-for-search-scoring,Relevance scoring>>
|8.18/9.0
|Calculate and sort by relevance with `METADATA _score`

|Enhanced match options
|8.18/9.0
|Configure text searches with additional parameters for the `match` function

|<<esql-for-search-kql,Kibana Query Language>>
|8.18/9.0
|Use Kibana Query Language with <<esql-kql,`kql`>> function

|<<esql-for-search-semantic,Semantic search>>
|8.18/9.0
|Perform semantic searches on `semantic_text` field types

|<<esql-for-search-hybrid,Hybrid search>>
|8.18/9.0
|Combine lexical and semantic search approaches with custom weights
|===

[[esql-filtering-vs-searching]]
==== Filtering vs. searching

{esql} can be used for both simple filtering and relevance-based searching:

* **Filtering** removes non-matching documents without calculating relevance scores
* **Searching** both filters documents and ranks them by how well they match the query

Note that filtering is faster than searching, because it doesn't require score calculations.

[[esql-for-search-scoring]]
===== Relevance scoring

To get the most relevant results first, you need to use `METADATA _score` and sort by score. For example:

[source,esql]
----
FROM books METADATA _score
| WHERE match(title, "Shakespeare") OR match(plot, "Shakespeare")
| SORT _score DESC
----

[[esql-for-search-how-scoring-works]]
===== How `_score` works

When working with relevance scoring in ES|QL:

* If you don't include `METADATA _score` in your query, this only performs filtering operations with no relevance calculation.
* When you include `METADATA _score`, any search function included in `WHERE` conditions contribute to the relevance score. This means that every occurrence of `MATCH`, `QSTR` and `KQL` will affect the score.
* Filtering operations that are not search functions, like range conditions and exact matches, don't affect the score.
* Including `METADATA _score` doesn't automatically sort your results by relevance. You must explicitly use `SORT _score DESC` or `SORT _score ASC` to order your results by relevance.

[[esql-for-search-full-text]]
==== Full text search

[[esql-for-search-match-function-operator]]
===== Match function and operator

ES|QL offers two syntax options for `match`, which replicate the functionality of <<query-dsl-match-query, `match`>> queries in Query DSL.

Use the compact operator syntax (`:`) for simple text matching with default parameters.

[source,esql]
----
FROM logs | WHERE match(message, "connection error")
----

Use the `match()` function syntax when you need to pass additional parameters:

[source,esql]
----
FROM products | WHERE match(name, "laptop", { "boost": 2.0 })
----

These full-text functions address several key limitations that existed for text filtering in {esql}:

* They work directly on multivalued fields, returning results when any value in a multivalued field matches the query
* They leverage analyzers, ensuring the query is analyzed with the same process as the indexed data (enabling case-insensitive matching, ASCII folding, stopword removal, and synonym support)
* They are highly performant, using Lucene index structures rather than pattern matching or regular expressions to locate terms in your data

Refer to this blog for more context: https://www.elastic.co/search-labs/blog/filtering-in-esql-full-text-search-match-qstr[Introducing full text filtering in ES|QL].

[TIP]
====
See <<match-field-params,Match field parameters>> for more advanced options using match. 
====

[IMPORTANT]
====
These queries match documents but don't automatically sort by relevance. To get the most relevant results first, you need to use `METADATA _score` and sort by score. See <<esql-for-search-scoring,Relevance scoring>> for more information.
====

[[esql-for-search-query-string]]
===== Query string function (`QSTR`)

The <<esql-qstr,`qstr` function>> provides the same functionality as the Query DSL's `query_string` query. This is for advanced use cases, such as wildcard searches, searches across multiple fields, and more.

[source,esql]
----
FROM articles METADATA _score
| WHERE QSTR("(new york city) OR (big apple)")
| SORT _score DESC
| LIMIT 10
----

For complete details, refer to the <<query-dsl-query-string-query, Query DSL `query_string` docs>>.

[[esql-for-search-kql]]
===== Kibana Query Language function (`KQL`)

Use the <<esql-kql,KQL function>> to use the {kibana-ref}/kuery-query.html[Kibana Query Language] in your ES|QL queries:

[source,esql]
----
FROM logs*
| WHERE KQL("http.request.method:GET AND agent.type:filebeat")
----

The `kql` function is useful when transitioning queries from Kibana's Discover, Dashboard, or other interfaces that use KQL. This will allow you to gradually migrate queries to ES|QL without needing to rewrite them all at once.

[[esql-for-search-semantic]]
==== Semantic search

You can perform semantic searches over <<semantic-text, `semantic_text`>> field types using the same match syntax as full-text search.

This example uses the match operator `:`:

[source,esql]
----
FROM articles METADATA _score
| WHERE semantic_content:"What are the impacts of climate change on agriculture?"
| SORT _score DESC
----

This example uses the match function:

[source,esql]
----
FROM articles METADATA _score
| WHERE match(semantic_content, "What are the impacts of climate change on agriculture?")
| SORT _score DESC
----

[[esql-for-search-hybrid]]
==== Hybrid search

Combine traditional and semantic search with custom weights:

[source,esql]
----
FROM books METADATA _score
| WHERE match(semantic_title, "fantasy adventure", { "boost": 0.75 }) 
    OR match(title, "fantasy adventure", { "boost": 0.25 })
| SORT _score DESC
----

[[esql-for-search-limitations]]
==== Limitations

Refer to <<esql-limitations-full-text-search, {esql} limitations>> for a list of known limitations.

[[esql-for-search-next-steps]]
==== Next steps

[[esql-for-search-tutorials]]
===== Tutorials and how-to guides

* <<esql-search-tutorial>>: Hands-on tutorial for getting started with search tools in {esql}
* <<semantic-search-semantic-text>>: Learn how to use the `semantic_text` field type

[[esql-for-search-reference]]
===== Technical reference

* <<esql-search-functions>>: Complete reference for all search functions
* <<esql-limitations-full-text-search, Limitations>>: Current limitations for search in ES|QL

[[esql-for-search-concepts]]
===== Background concepts

* <<analysis>>: Learn how text is processed for full-text search
* <<semantic-search>>: Get an overview of semantic search in {es}
* <<query-filter-context>>: Understand the difference between query and filter contexts in {es}

[[esql-for-search-blogs]]
===== Related blog posts

// TODO* https://www.elastic.co/blog/esql-you-know-for-search-scoring-semantic-search[ES|QL, you know for Search]: Introducing scoring and semantic search
* https://www.elastic.co/search-labs/blog/filtering-in-esql-full-text-search-match-qstr[Introducing full text filtering in ES|QL]: Overview of text filtering capabilities