[[esql-for-search]] === Using {esql} for search preview::[] This page provides an overview of how to use {esql} for search use cases. [TIP] ==== Prefer to get started with a hands-on tutorial? Check out <>. ==== The following table summarizes the key search features available in {esql} and when they were introduced. [cols="1,1,2", options="header"] |=== |Feature |Available since |Description |<> |8.17 |Perform basic text searches with <> and <> |<> |8.17 |Execute complex queries with <> using Query String syntax |<> |8.18/9.0 |Calculate and sort by relevance with `METADATA _score` |Enhanced match options |8.18/9.0 |Configure text searches with additional parameters for the `match` function |<> |8.18/9.0 |Use Kibana Query Language with <> function |<> |8.18/9.0 |Perform semantic searches on `semantic_text` field types |<> |8.18/9.0 |Combine lexical and semantic search approaches with custom weights |=== [[esql-filtering-vs-searching]] ==== Filtering vs. searching {esql} can be used for both simple filtering and relevance-based searching: * **Filtering** removes non-matching documents without calculating relevance scores * **Searching** both filters documents and ranks them by how well they match the query Note that filtering is faster than searching, because it doesn't require score calculations. [[esql-for-search-scoring]] ===== Relevance scoring To get the most relevant results first, you need to use `METADATA _score` and sort by score. For example: [source,esql] ---- FROM books METADATA _score | WHERE match(title, "Shakespeare") OR match(plot, "Shakespeare") | SORT _score DESC ---- [[esql-for-search-how-scoring-works]] ===== How `_score` works When working with relevance scoring in ES|QL: * If you don't include `METADATA _score` in your query, this only performs filtering operations with no relevance calculation. * When you include `METADATA _score`, any search function included in `WHERE` conditions contribute to the relevance score. This means that every occurrence of `MATCH`, `QSTR` and `KQL` will affect the score. * Filtering operations that are not search functions, like range conditions and exact matches, don't affect the score. * Including `METADATA _score` doesn't automatically sort your results by relevance. You must explicitly use `SORT _score DESC` or `SORT _score ASC` to order your results by relevance. [[esql-for-search-full-text]] ==== Full text search [[esql-for-search-match-function-operator]] ===== Match function and operator ES|QL offers two syntax options for `match`, which replicate the functionality of <> queries in Query DSL. Use the compact operator syntax (`:`) for simple text matching with default parameters. [source,esql] ---- FROM logs | WHERE match(message, "connection error") ---- Use the `match()` function syntax when you need to pass additional parameters: [source,esql] ---- FROM products | WHERE match(name, "laptop", { "boost": 2.0 }) ---- These full-text functions address several key limitations that existed for text filtering in {esql}: * They work directly on multivalued fields, returning results when any value in a multivalued field matches the query * They leverage analyzers, ensuring the query is analyzed with the same process as the indexed data (enabling case-insensitive matching, ASCII folding, stopword removal, and synonym support) * They are highly performant, using Lucene index structures rather than pattern matching or regular expressions to locate terms in your data Refer to this blog for more context: https://www.elastic.co/search-labs/blog/filtering-in-esql-full-text-search-match-qstr[Introducing full text filtering in ES|QL]. [TIP] ==== See <> for more advanced options using match. ==== [IMPORTANT] ==== These queries match documents but don't automatically sort by relevance. To get the most relevant results first, you need to use `METADATA _score` and sort by score. See <> for more information. ==== [[esql-for-search-query-string]] ===== Query string function (`QSTR`) The <> provides the same functionality as the Query DSL's `query_string` query. This is for advanced use cases, such as wildcard searches, searches across multiple fields, and more. [source,esql] ---- FROM articles METADATA _score | WHERE QSTR("(new york city) OR (big apple)") | SORT _score DESC | LIMIT 10 ---- For complete details, refer to the <>. [[esql-for-search-kql]] ===== Kibana Query Language function (`KQL`) Use the <> to use the {kibana-ref}/kuery-query.html[Kibana Query Language] in your ES|QL queries: [source,esql] ---- FROM logs* | WHERE KQL("http.request.method:GET AND agent.type:filebeat") ---- The `kql` function is useful when transitioning queries from Kibana's Discover, Dashboard, or other interfaces that use KQL. This will allow you to gradually migrate queries to ES|QL without needing to rewrite them all at once. [[esql-for-search-semantic]] ==== Semantic search You can perform semantic searches over <> field types using the same match syntax as full-text search. This example uses the match operator `:`: [source,esql] ---- FROM articles METADATA _score | WHERE semantic_content:"What are the impacts of climate change on agriculture?" | SORT _score DESC ---- This example uses the match function: [source,esql] ---- FROM articles METADATA _score | WHERE match(semantic_content, "What are the impacts of climate change on agriculture?") | SORT _score DESC ---- [[esql-for-search-hybrid]] ==== Hybrid search Combine traditional and semantic search with custom weights: [source,esql] ---- FROM books METADATA _score | WHERE match(semantic_title, "fantasy adventure", { "boost": 0.75 }) OR match(title, "fantasy adventure", { "boost": 0.25 }) | SORT _score DESC ---- [[esql-for-search-limitations]] ==== Limitations Refer to <> for a list of known limitations. [[esql-for-search-next-steps]] ==== Next steps [[esql-for-search-tutorials]] ===== Tutorials and how-to guides * <>: Hands-on tutorial for getting started with search tools in {esql} * <>: Learn how to use the `semantic_text` field type [[esql-for-search-reference]] ===== Technical reference * <>: Complete reference for all search functions * <>: Current limitations for search in ES|QL [[esql-for-search-concepts]] ===== Background concepts * <>: Learn how text is processed for full-text search * <>: Get an overview of semantic search in {es} * <>: Understand the difference between query and filter contexts in {es} [[esql-for-search-blogs]] ===== Related blog posts * https://www.elastic.co/search-labs/blog/esql-introducing-scoring-semantic-search[ES|QL, you know for Search]: Introducing scoring and semantic search * https://www.elastic.co/search-labs/blog/filtering-in-esql-full-text-search-match-qstr[Introducing full text filtering in ES|QL]: Overview of text filtering capabilities