12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034 |
- [role="xpack"]
- [testenv="basic"]
- [[eql-syntax]]
- == EQL syntax reference
- ++++
- <titleabbrev>Syntax reference</titleabbrev>
- ++++
- [discrete]
- [[eql-basic-syntax]]
- === Basic syntax
- EQL queries require an event category and a matching condition. The `where`
- keyword connects them.
- [source,eql]
- ----
- event_category where condition
- ----
- For example, the following EQL query matches `process` events with a
- `process.name` field value of `svchost.exe`:
- [source,eql]
- ----
- process where process.name == "svchost.exe"
- ----
- [discrete]
- [[eql-syntax-event-categories]]
- === Event categories
- An event category is a valid, indexed value of the
- <<eql-required-fields,event category field>>. You can set the event category
- field using the `event_category_field` parameter of the EQL search API.
- [discrete]
- [[eql-syntax-match-any-event-category]]
- === Match any event category
- To match events of any category, use the `any` keyword. You can also use the
- `any` keyword to search for documents without a event category field.
- For example, the following EQL query matches any documents with a
- `network.protocol` field value of `http`:
- [source,eql]
- ----
- any where network.protocol == "http"
- ----
- [discrete]
- [[eql-syntax-escape-an-event-category]]
- === Escape an event category
- Use enclosing double quotes (`"`) or three enclosing double quotes (`"""`) to
- escape event categories that:
- * Contain a special character, such as a hyphen (`-`) or dot (`.`)
- * Contain a space
- * Start with a numeral
- [source,eql]
- ----
- ".my.event.category"
- "my-event-category"
- "my event category"
- "6eventcategory"
- """.my.event.category"""
- """my-event-category"""
- """my event category"""
- """6eventcategory"""
- ----
- [discrete]
- [[eql-syntax-escape-a-field-name]]
- === Escape a field name
- Use enclosing enclosing backticks (+++`+++) to escape field names that:
- * Contain a hyphen (`-`)
- * Contain a space
- * Start with a numeral
- [source,eql]
- ----
- `my-field`
- `my field`
- `6myfield`
- ----
- Use double backticks (+++``+++) to escape any backticks (+++`+++) in the field
- name.
- [source,eql]
- ----
- my`field -> `my``field`
- ----
- [discrete]
- [[eql-syntax-conditions]]
- === Conditions
- A condition consists of one or more criteria an event must match.
- You can specify and combine these criteria using the following operators. Most
- EQL operators are case-sensitive by default.
- [discrete]
- [[eql-syntax-comparison-operators]]
- === Comparison operators
- [source,eql]
- ----
- < <= == : != >= >
- ----
- `<` (less than)::
- Returns `true` if the value to the left of the operator is less than the value
- to the right. Otherwise returns `false`.
- `<=` (less than or equal) ::
- Returns `true` if the value to the left of the operator is less than or equal to
- the value to the right. Otherwise returns `false`.
- `==` (equal, case-sensitive)::
- Returns `true` if the values to the left and right of the operator are equal.
- Otherwise returns `false`. For strings, matching is case-sensitive. Wildcards
- are not supported.
- `:` (equal, case-insensitive)::
- Returns `true` if strings to the left and right of the operator are equal.
- Otherwise returns `false`. Matching is case-insensitive and can only be used to
- compare strings. Supports <<eql-syntax-wildcards,wildcards>> and
- <<eql-syntax-lookup-operators,list lookups>>.
- [IMPORTANT]
- ====
- Avoid using the `==` or `:` operators to perform exact matching on
- <<text,`text`>> field values.
- By default, {es} changes the values of `text` fields as part of <<analysis,
- analysis>>. This can make finding exact matches for `text` field values
- difficult.
- To search `text` fields, consider using a <<eql-search-filter-query-dsl,query
- DSL filter>> that contains a <<query-dsl-match-query,`match`>> query.
- ====
- `!=` (not equal, case-sensitive)::
- Returns `true` if the values to the left and right of the operator are not
- equal. Otherwise returns `false`. For strings, matching is case-sensitive.
- Wildcards are not supported.
- `>=` (greater than or equal) ::
- Returns `true` if the value to the left of the operator is greater than or equal
- to the value to the right. Otherwise returns `false`. When comparing strings,
- the operator uses a case-sensitive lexicographic order.
- `>` (greater than)::
- Returns `true` if the value to the left of the operator is greater than the
- value to the right. Otherwise returns `false`. When comparing strings,
- the operator uses a case-sensitive lexicographic order.
- NOTE: `=` is not supported as an equal operator. Use `==` or `:` instead.
- You cannot chain comparison operators. Instead, use a
- <<eql-syntax-logical-operators,logical operator>> between comparisons. For
- example, `foo < bar <= baz` is not supported. However, you can rewrite the
- expression as `foo < bar and bar <= baz`, which is supported.
- You also cannot use comparison operators to compare a field to another field.
- This applies even if the fields are changed using a <<eql-functions,function>>.
- *Example* +
- The following EQL query compares the `process.parent_name` field
- value to a static value, `foo`. This comparison is supported.
- However, the query also compares the `process.parent.name` field value to the
- `process.name` field. This comparison is not supported and will return an
- error for the entire query.
- [source,eql]
- ----
- process where process.parent.name == "foo" and process.parent.name == process.name
- ----
- Instead, you can rewrite the query to compare both the `process.parent.name`
- and `process.name` fields to static values.
- [source,eql]
- ----
- process where process.parent.name == "foo" and process.name == "foo"
- ----
- [discrete]
- [[eql-syntax-logical-operators]]
- === Logical operators
- [source,eql]
- ----
- and or not
- ----
- `and`::
- Returns `true` only if the condition to the left and right _both_ return `true`.
- Otherwise returns `false`.
- `or`::
- Returns `true` if one of the conditions to the left or right `true`.
- Otherwise returns `false`.
- `not`::
- Returns `true` if the condition to the right is `false`.
- [discrete]
- [[eql-syntax-lookup-operators]]
- === Lookup operators
- [source,eql]
- ----
- user.name in ("Administrator", "SYSTEM", "NETWORK SERVICE")
- user.name in~ ("administrator", "system", "network service")
- user.name not in ("Administrator", "SYSTEM", "NETWORK SERVICE")
- user.name not in~ ("administrator", "system", "network service")
- user.name : ("administrator", "system", "network service")
- ----
- `in` (case-sensitive)::
- Returns `true` if the value is contained in the provided list. For strings,
- matching is case-sensitive.
- `in~` (case-insensitive)::
- Returns `true` if the value is contained in the provided list. For strings,
- matching is case-insensitive.
- `not in` (case-sensitive)::
- Returns `true` if the value is not contained in the provided list. For strings,
- matching is case-sensitive.
- `not in~` (case-insensitive)::
- Returns `true` if the value is not contained in the provided list. For strings,
- matching is case-insensitive.
- `:` (case-insensitive)::
- Returns `true` if the value is contained in the provided list. Can only be used
- to compare strings.
- [discrete]
- [[eql-syntax-math-operators]]
- === Math operators
- [source,eql]
- ----
- + - * / %
- ----
- `+` (add)::
- Adds the values to the left and right of the operator.
- `-` (subtract)::
- Subtracts the value to the right of the operator from the value to the left.
- `*` (multiply)::
- Multiplies the values to the left and right of the operator.
- `/` (divide)::
- Divides the value to the left of the operator by the value to the right.
- +
- [[eql-divide-operator-float-rounding]]
- [WARNING]
- ====
- If both the dividend and divisor are integers, the divide (`\`) operation
- _rounds down_ any returned floating point numbers to the nearest integer. To
- avoid rounding, convert either the dividend or divisor to a float.
- *Example* +
- The `process.args_count` field is a <<number,`long`>> integer field containing a
- count of process arguments.
- A user might expect the following EQL query to only match events with a
- `process.args_count` value of `4`.
- [source,eql]
- ----
- process where ( 4 / process.args_count ) == 1
- ----
- However, the EQL query matches events with a `process.args_count` value of `3`
- or `4`.
- For events with a `process.args_count` value of `3`, the divide operation
- returns a float of `1.333...`, which is rounded down to `1`.
- To match only events with a `process.args_count` value of `4`, convert
- either the dividend or divisor to a float.
- The following EQL query changes the integer `4` to the equivalent float `4.0`.
- [source,eql]
- ----
- process where ( 4.0 / process.args_count ) == 1
- ----
- ====
- `%` (modulo)::
- Divides the value to the left of the operator by the value to the right. Returns only the remainder.
- [discrete]
- [[eql-syntax-match-any-condition]]
- === Match any condition
- To match events solely on event category, use the `where true` condition.
- For example, the following EQL query matches any `file` events:
- [source,eql]
- ----
- file where true
- ----
- To match any event, you can combine the `any` keyword with the `where true`
- condition:
- [source,eql]
- ----
- any where true
- ----
- [discrete]
- [[eql-syntax-strings]]
- === Strings
- Strings are enclosed in double quotes (`"`).
- [source,eql]
- ----
- "hello world"
- ----
- Strings enclosed in single quotes (`'`) are not supported.
- [discrete]
- [[eql-syntax-escape-characters]]
- === Escape characters in a string
- When used within a string, special characters, such as a carriage return or
- double quote (`"`), must be escaped with a preceding backslash (`\`).
- [source,eql]
- ----
- "example \r of \" escaped \n characters"
- ----
- [options="header"]
- |====
- | Escape sequence | Literal character
- |`\n` | A newline (linefeed) character
- |`\r` | A carriage return character
- |`\t` | A tab character
- |`\\` | A backslash (`\`) character
- |`\"` | A double quote (`"`) character
- |====
- IMPORTANT: The single quote (`'`) character is reserved for future use. You
- cannot use an escaped single quote (`\'`) for literal strings. Use an escaped
- double quote (`\"`) instead.
- [discrete]
- [[eql-syntax-raw-strings]]
- === Raw strings
- Raw strings treat special characters, such as backslashes (`\`), as literal
- characters. Raw strings are enclosed in three double quotes (`"""`).
- [source,eql]
- ----
- """Raw string with a literal double quote " and blackslash \ included"""
- ----
- A raw string cannot contain three consecutive double quotes (`"""`). Instead,
- use a regular string with the `\"` escape sequence.
- [source,eql]
- ----
- "String containing \"\"\" three double quotes"
- ----
- [discrete]
- [[eql-syntax-wildcards]]
- === Wildcards
- For string comparisons using the `:` operator, you can use the `*` and `?`
- wildcards to match specific patterns. The `*` wildcard matches zero or more
- characters:
- [source,eql]
- ----
- my_field : "doc*" // Matches "doc", "docs", or "document" but not "dos"
- my_field : "*doc" // Matches "adoc" or "asciidoc"
- my_field : "d*c" // Matches "doc" or "disc"
- ----
- The `?` wildcard matches exactly one character:
- [source,eql]
- ----
- my_field : "doc?" // Matches "docs" but not "doc", "document", or "dos"
- my_field : "?doc" // Matches "adoc" but not "asciidoc"
- my_field : "d?c" // Matches "doc" but not "disc"
- ----
- The `:` operator also supports wildcards in <<eql-syntax-lookup-operators,list
- lookups>>:
- [source,eql]
- ----
- my_field : ("doc*", "f*o", "ba?", "qux")
- ----
- [discrete]
- [[eql-sequences]]
- === Sequences
- You can use EQL sequences to describe and match an ordered series of events.
- Each item in a sequence is an event category and event condition,
- surrounded by square brackets (`[ ]`). Events are listed in ascending
- chronological order, with the most recent event listed last.
- [source,eql]
- ----
- sequence
- [ event_category_1 where condition_1 ]
- [ event_category_2 where condition_2 ]
- ...
- ----
- *Example* +
- The following EQL sequence query matches this series of ordered events:
- . Start with an event with:
- +
- --
- * An event category of `file`
- * A `file.extension` of `exe`
- --
- . Followed by an event with an event category of `process`
- [source,eql]
- ----
- sequence
- [ file where file.extension == "exe" ]
- [ process where true ]
- ----
- [discrete]
- [[eql-with-maxspan-keywords]]
- === `with maxspan` keywords
- You can use the `with maxspan` keywords to constrain a sequence to a specified
- timespan. All events in a matching sequence must occur within this duration,
- starting at the first event's timestamp.
- The `maxspan` keyword accepts <<time-units,time value>> arguments.
- [source,eql]
- ----
- sequence with maxspan=30s
- [ event_category_1 where condition_1 ] by field_baz
- [ event_category_2 where condition_2 ] by field_bar
- ...
- ----
- *Example* +
- The following sequence query uses a `maxspan` value of `15m` (15 minutes).
- Events in a matching sequence must occur within 15 minutes of the first event's
- timestamp.
- [source,eql]
- ----
- sequence with maxspan=15m
- [ file where file.extension == "exe" ]
- [ process where true ]
- ----
- [discrete]
- [[eql-by-keyword]]
- === `by` keyword
- You can use the `by` keyword with sequences to only match events that share the
- same field values. If a field value should be shared across all events, you
- can use `sequence by`.
- [source,eql]
- ----
- sequence by field_foo
- [ event_category_1 where condition_1 ] by field_baz
- [ event_category_2 where condition_2 ] by field_bar
- ...
- ----
- *Example* +
- The following sequence query uses the `by` keyword to constrain matching events
- to:
- * Events with the same `user.name` value
- * `file` events with a `file.path` value equal to the following `process`
- event's `process.path` value.
- [source,eql]
- ----
- sequence
- [ file where file.extension == "exe" ] by user.name, file.path
- [ process where true ] by user.name, process.path
- ----
- Because the `user.name` field is shared across all events in the sequence, it
- can be included using `sequence by`. The following sequence is equivalent to the
- prior one.
- [source,eql]
- ----
- sequence by user.name
- [ file where file.extension == "exe" ] by file.path
- [ process where true ] by process.path
- ----
- You can combine the `sequence by` and `with maxspan` keywords to constrain a
- sequence by both field values and a timespan.
- [source,eql]
- ----
- sequence by field_foo with maxspan=30s
- [ event_category_1 where condition_1 ] by field_baz
- [ event_category_2 where condition_2 ] by field_bar
- ...
- ----
- *Example* +
- The following sequence query uses the `sequence by` keyword and `with maxspan`
- keywords to match only a sequence of events that:
- * Share the same `user.name` field values
- * Occur within `15m` (15 minutes) of the first matching event
- [source,eql]
- ----
- sequence by user.name with maxspan=15m
- [ file where file.extension == "exe" ] by file.path
- [ process where true ] by process.path
- ----
- [discrete]
- [[eql-until-keyword]]
- === `until` keyword
- You can use the `until` keyword to specify an expiration event for a sequence.
- If this expiration event occurs _between_ matching events in a sequence, the
- sequence expires and is not considered a match. If the expiration event occurs
- _after_ matching events in a sequence, the sequence is still considered a
- match. The expiration event is not included in the results.
- [source,eql]
- ----
- sequence
- [ event_category_1 where condition_1 ]
- [ event_category_2 where condition_2 ]
- ...
- until [ event_category_3 where condition_3 ]
- ----
- *Example* +
- A dataset contains the following event sequences, grouped by shared IDs:
- [source,txt]
- ----
- A, B
- A, B, C
- A, C, B
- ----
- The following EQL query searches the dataset for sequences containing
- event `A` followed by event `B`. Event `C` is used as an expiration event.
- [source,eql]
- ----
- sequence by ID
- A
- B
- until C
- ----
- The query matches sequences `A, B` and `A, B, C` but not `A, C, B`.
- [TIP]
- ====
- The `until` keyword can be useful when searching for process sequences in
- Windows event logs.
- In Windows, a process ID (PID) is unique only while a process is running. After
- a process terminates, its PID can be reused.
- You can search for a sequence of events with the same PID value using the `by`
- and `sequence by` keywords.
- *Example* +
- The following EQL query uses the `sequence by` keyword to match a
- sequence of events that share the same `process.pid` value.
- [source,eql]
- ----
- sequence by process.pid
- [ process where event.type == "start" and process.name == "cmd.exe" ]
- [ process where file.extension == "exe" ]
- ----
- However, due to PID reuse, this can result in a matching sequence that
- contains events across unrelated processes. To prevent false positives, you can
- use the `until` keyword to end matching sequences before a process termination
- event.
- The following EQL query uses the `until` keyword to end sequences before
- `process` events with an `event.type` of `stop`. These events indicate a process
- has been terminated.
- [source,eql]
- ----
- sequence by process.pid
- [ process where event.type == "start" and process.name == "cmd.exe" ]
- [ process where file.extension == "exe" ]
- until [ process where event.type == "stop" ]
- ----
- ====
- [discrete]
- [[eql-functions]]
- === Functions
- You can use EQL functions to convert data types, perform math, manipulate
- strings, and more. For a list of supported functions, see <<eql-function-ref>>.
- [discrete]
- [[eql-case-insensitive-functions]]
- === Case-insensitive functions
- Most EQL functions are case-sensitive by default. To make a function
- case-insensitive, use the `~` operator after the function name:
- [source,eql]
- ----
- stringContains(process.name,".exe") // Matches ".exe" but not ".EXE" or ".Exe"
- stringContains~(process.name,".exe") // Matches ".exe", ".EXE", or ".Exe"
- ----
- [discrete]
- [[eql-how-functions-impact-search-performance]]
- === How functions impact search performance
- Using functions in EQL queries can result in slower search speeds. If you
- often use functions to transform indexed data, you can speed up search by making
- these changes during indexing instead. However, that often means slower index
- speeds.
- *Example* +
- An index contains the `file.path` field. `file.path` contains the full path to a
- file, including the file extension.
- When running EQL searches, users often use the `endsWith` function with the
- `file.path` field to match file extensions:
- [source,eql]
- ----
- file where endsWith(file.path,".exe") or endsWith(file.path,".dll")
- ----
- While this works, it can be repetitive to write and can slow search speeds. To
- speed up search, you can do the following instead:
- . <<indices-put-mapping,Add a new field>>, `file.extension`, to the index. The
- `file.extension` field will contain only the file extension from the
- `file.path` field.
- . Use an <<ingest,ingest pipeline>> containing the <<grok-processor,`grok`>>
- processor or another preprocessor tool to extract the file extension from the
- `file.path` field before indexing.
- . Index the extracted file extension to the `file.extension` field.
- These changes may slow indexing but allow for faster searches. Users
- can use the `file.extension` field instead of multiple `endsWith` function
- calls:
- [source,eql]
- ----
- file where file.extension in ("exe", "dll")
- ----
- We recommend testing and benchmarking any indexing changes before deploying them
- in production. See <<tune-for-indexing-speed>> and <<tune-for-search-speed>>.
- [discrete]
- [[eql-pipes]]
- === Pipes
- EQL pipes filter, aggregate, and post-process events returned by
- an EQL query. You can use pipes to narrow down EQL query results or make them
- more specific.
- Pipes are delimited using the pipe (`|`) character.
- [source,eql]
- ----
- event_category where condition | pipe
- ----
- *Example* +
- The following EQL query uses the `tail` pipe to return only the 10 most recent
- events matching the query.
- [source,eql]
- ----
- authentication where agent.id == 4624
- | tail 10
- ----
- You can pass the output of a pipe to another pipe. This lets you use multiple
- pipes with a single query.
- For a list of supported pipes, see <<eql-pipe-ref>>.
- [discrete]
- [[eql-syntax-limitations]]
- === Limitations
- EQL does not support the following features and syntax.
- [discrete]
- [[eql-compare-fields]]
- ==== Comparing fields
- You cannot use EQL comparison operators to compare a field to
- another field. This applies even if the fields are changed using a
- <<eql-functions,function>>.
- [discrete]
- [[eql-array-fields]]
- ==== Array field values are not supported
- EQL does not support <<array,array>> field values, also known as
- multi-value fields. EQL searches on array field values may return inconsistent
- results.
- [discrete]
- [[eql-nested-fields]]
- ==== EQL search on nested fields
- You cannot use EQL to search the values of a <<nested,`nested`>> field or the
- sub-fields of a `nested` field. However, data streams and indices containing
- `nested` field mappings are otherwise supported.
- [discrete]
- [[eql-unsupported-syntax]]
- ==== Differences from Endgame EQL syntax
- {es} EQL differs from the {eql-ref}/index.html[Elastic Endgame EQL syntax] as
- follows:
- * Most operators and functions in {es} EQL are case-sensitive by default. For
- case-insensitive equality comparisons, use the `:` operator. To make a function
- case-insensitive, use the `~` operator after the function name. See
- <<eql-case-insensitive-functions>>.
- * Comparisons using the `==` and `!=` operators do not expand wildcard
- characters. For example, `process_name == "cmd*.exe"` interprets `*` as a
- literal asterisk, not a wildcard. For case-sensitive wildcard matching, use the
- <<eql-fn-wildcard,`wildcard`>> function.
- * `=` cannot be substituted for the `==` operator.
- * Strings enclosed in single quotes (`'`) are not supported. Enclose strings in
- double quotes (`"`) instead.
- * `?"` and `?'` do not indicate raw strings. Enclose raw strings in
- three double quotes (`"""`) instead.
- * {es} EQL does not support:
- ** Array functions:
- *** {eql-ref}/functions.html#arrayContains[`arrayContains`]
- *** {eql-ref}/functions.html#arrayCount[`arrayCount`]
- *** {eql-ref}/functions.html#arraySearch[`arraySearch`]
- ** The {eql-ref}//functions.html#match[`match`] function
- ** {eql-ref}/joins.html[Joins]
- ** {eql-ref}/basic-syntax.html#event-relationships[Lineage-related keywords]:
- *** `child of`
- *** `descendant of`
- *** `event of`
- ** The following {eql-ref}/pipes.html[pipes]:
- *** {eql-ref}/pipes.html#count[`count`]
- *** {eql-ref}/pipes.html#filter[`filter`]
- *** {eql-ref}/pipes.html#sort[`sort`]
- *** {eql-ref}/pipes.html#unique[`unique`]
- *** {eql-ref}/pipes.html#unique-count[`unique_count`]
- [discrete]
- [[eql-how-sequence-queries-handle-matches]]
- ==== How sequence queries handle matches
- <<eql-sequences,Sequence queries>> don't find all potential matches for a
- sequence. This approach would be too slow and costly for large event data sets.
- Instead, a sequence query handles pending sequence matches as a
- {wikipedia}/Finite-state_machine[state machine]:
- * Each event item in the sequence query is a state in the machine.
- * Only one pending sequence can be in each state at a time.
- * If two pending sequences are in the same state at the same time, the most
- recent sequence overwrites the older one.
- * If the query includes <<eql-by-keyword,`by` fields>>, the query uses a
- separate state machine for each unique `by` field value.
- .*Example*
- [%collapsible]
- ====
- A data set contains the following `process` events in ascending chronological
- order:
- [source,js]
- ----
- { "index" : { "_id": "1" } }
- { "user": { "name": "root" }, "process": { "name": "attrib" }, ...}
- { "index" : { "_id": "2" } }
- { "user": { "name": "root" }, "process": { "name": "attrib" }, ...}
- { "index" : { "_id": "3" } }
- { "user": { "name": "elkbee" }, "process": { "name": "bash" }, ...}
- { "index" : { "_id": "4" } }
- { "user": { "name": "root" }, "process": { "name": "bash" }, ...}
- { "index" : { "_id": "5" } }
- { "user": { "name": "root" }, "process": { "name": "bash" }, ...}
- { "index" : { "_id": "6" } }
- { "user": { "name": "elkbee" }, "process": { "name": "attrib" }, ...}
- { "index" : { "_id": "7" } }
- { "user": { "name": "root" }, "process": { "name": "attrib" }, ...}
- { "index" : { "_id": "8" } }
- { "user": { "name": "elkbee" }, "process": { "name": "bash" }, ...}
- { "index" : { "_id": "9" } }
- { "user": { "name": "root" }, "process": { "name": "cat" }, ...}
- { "index" : { "_id": "10" } }
- { "user": { "name": "elkbee" }, "process": { "name": "cat" }, ...}
- { "index" : { "_id": "11" } }
- { "user": { "name": "root" }, "process": { "name": "cat" }, ...}
- ----
- // NOTCONSOLE
- An EQL sequence query searches the data set:
- [source,eql]
- ----
- sequence by user.name
- [process where process.name == "attrib"]
- [process where process.name == "bash"]
- [process where process.name == "cat"]
- ----
- The query's event items correspond to the following states:
- * State A: `[process where process.name == "attrib"]`
- * State B: `[process where process.name == "bash"]`
- * Complete: `[process where process.name == "cat"]`
- image::images/eql/sequence-state-machine.svg[align="center"]
- To find matching sequences, the query uses separate state machines for each
- unique `user.name` value. Based on the data set, you can expect two state
- machines: one for the `root` user and one for `elkbee`.
- image::images/eql/separate-state-machines.svg[align="center"]
- Pending sequence matches move through each machine's states as follows:
- [source,txt]
- ----
- { "index" : { "_id": "1" } }
- { "user": { "name": "root" }, "process": { "name": "attrib" }, ...}
- // Creates sequence [1] in state A for the "root" user.
- //
- // +------------------------"root"------------------------+
- // | +-----------+ +-----------+ +------------+ |
- // | | State A | | State B | | Complete | |
- // | +-----------+ +-----------+ +------------+ |
- // | | [1] | | | | | |
- // | +-----------+ +-----------+ +------------+ |
- // +------------------------------------------------------+
- { "index" : { "_id": "2" } }
- { "user": { "name": "root" }, "process": { "name": "attrib" }, ...}
- // Creates sequence [2] in state A for "root", overwriting sequence [1].
- //
- // +------------------------"root"------------------------+
- // | +-----------+ +-----------+ +------------+ |
- // | | State A | | State B | | Complete | |
- // | +-----------+ +-----------+ +------------+ |
- // | | [2] | | | | | |
- // | +-----------+ +-----------+ +------------+ |
- // +------------------------------------------------------+
- { "index" : { "_id": "3" } }
- { "user": { "name": "elkbee" }, "process": { "name": "bash" }, ...}
- // Nothing happens. The "elkbee" user has no pending sequence to move
- // from state A to state B.
- //
- // +-----------------------"elkbee"-----------------------+
- // | +-----------+ +-----------+ +------------+ |
- // | | State A | | State B | | Complete | |
- // | +-----------+ +-----------+ +------------+ |
- // | | | | | | | |
- // | +-----------+ +-----------+ +------------+ |
- // +------------------------------------------------------+
- { "index" : { "_id": "4" } }
- { "user": { "name": "root" }, "process": { "name": "bash" }, ...}
- // Sequence [2] moves out of state A for "root".
- // State B for "root" now contains [2, 4].
- // State A for "root" is empty.
- //
- // +------------------------"root"------------------------+
- // | +-----------+ +-----------+ +------------+ |
- // | | State A | | State B | | Complete | |
- // | +-----------+ --> +-----------+ +------------+ |
- // | | | | [2, 4] | | | |
- // | +-----------+ +-----------+ +------------+ |
- // +------------------------------------------------------+
- { "index" : { "_id": "5" } }
- { "user": { "name": "root" }, "process": { "name": "bash" }, ...}
- // Nothing happens. State A is empty for "root".
- //
- // +------------------------"root"------------------------+
- // | +-----------+ +-----------+ +------------+ |
- // | | State A | | State B | | Complete | |
- // | +-----------+ +-----------+ +------------+ |
- // | | | | [2, 4] | | | |
- // | +-----------+ +-----------+ +------------+ |
- // +------------------------------------------------------+
- { "index" : { "_id": "6" } }
- { "user": { "name": "elkbee" }, "process": { "name": "attrib" }, ...}
- // Creates sequence [6] in state A for "elkbee".
- //
- // +-----------------------"elkbee"-----------------------+
- // | +-----------+ +-----------+ +------------+ |
- // | | State A | | State B | | Complete | |
- // | +-----------+ +-----------+ +------------+ |
- // | | [6] | | | | | |
- // | +-----------+ +-----------+ +------------+ |
- // +------------------------------------------------------+
- { "index" : { "_id": "7" } }
- { "user": { "name": "root" }, "process": { "name": "attrib" }, ...}
- // Creates sequence [7] in state A for "root".
- // Sequence [2, 4] remains in state B for "root".
- //
- // +------------------------"root"------------------------+
- // | +-----------+ +-----------+ +------------+ |
- // | | State A | | State B | | Complete | |
- // | +-----------+ +-----------+ +------------+ |
- // | | [7] | | [2, 4] | | | |
- // | +-----------+ +-----------+ +------------+ |
- // +------------------------------------------------------+
- { "index" : { "_id": "8" } }
- { "user": { "name": "elkbee" }, "process": { "name": "bash" }, ...}
- // Sequence [6, 8] moves to state B for "elkbee".
- // State A for "elkbee" is now empty.
- //
- // +-----------------------"elkbee"-----------------------+
- // | +-----------+ +-----------+ +------------+ |
- // | | State A | | State B | | Complete | |
- // | +-----------+ --> +-----------+ +------------+ |
- // | | | | [6, 8] | | | |
- // | +-----------+ +-----------+ +------------+ |
- // +------------------------------------------------------+
- { "index" : { "_id": "9" } }
- { "user": { "name": "root" }, "process": { "name": "cat" }, ...}
- // Sequence [2, 4, 9] is complete for "root".
- // State B for "root" is now empty.
- // Sequence [7] remains in state A.
- //
- // +------------------------"root"------------------------+
- // | +-----------+ +-----------+ +------------+ |
- // | | State A | | State B | | Complete | |
- // | +-----------+ +-----------+ --> +------------+ |
- // | | [7] | | | | [2, 4, 9] |
- // | +-----------+ +-----------+ +------------+ |
- // +------------------------------------------------------+
- { "index" : { "_id": "10" } }
- { "user": { "name": "elkbee" }, "process": { "name": "cat" }, ...}
- // Sequence [6, 8, 10] is complete for "elkbee".
- // State A and B for "elkbee" are now empty.
- //
- // +-----------------------"elkbee"-----------------------+
- // | +-----------+ +-----------+ +------------+ |
- // | | State A | | State B | | Complete | |
- // | +-----------+ +-----------+ --> +------------+ |
- // | | | | | | [6, 8, 10] |
- // | +-----------+ +-----------+ +------------+ |
- // +------------------------------------------------------+
- { "index" : { "_id": "11" } }
- { "user": { "name": "root" }, "process": { "name": "cat" }, ...}
- // Nothing happens.
- // The machines for "root" and "elkbee" remain the same.
- //
- // +------------------------"root"------------------------+
- // | +-----------+ +-----------+ +------------+ |
- // | | State A | | State B | | Complete | |
- // | +-----------+ +-----------+ +------------+ |
- // | | [7] | | | | [2, 4, 9] |
- // | +-----------+ +-----------+ +------------+ |
- // +------------------------------------------------------+
- //
- // +-----------------------"elkbee"-----------------------+
- // | +-----------+ +-----------+ +------------+ |
- // | | State A | | State B | | Complete | |
- // | +-----------+ +-----------+ +------------+ |
- // | | | | | | [6, 8, 10] |
- // | +-----------+ +-----------+ +------------+ |
- // +------------------------------------------------------+
- ----
- ====
|