Browse Source

[DOCS] Network community ID processor for ingest pipelines (#66592)

Dan Hermann 4 years ago
parent
commit
427d135cb6

+ 1 - 0
docs/reference/ingest/ingest-node.asciidoc

@@ -870,6 +870,7 @@ See {plugins}/ingest.html[Ingest plugins] for information about the available in
 include::processors/append.asciidoc[]
 include::processors/bytes.asciidoc[]
 include::processors/circle.asciidoc[]
+include::processors/community-id.asciidoc[]
 include::processors/convert.asciidoc[]
 include::processors/csv.asciidoc[]
 include::processors/date.asciidoc[]

+ 99 - 0
docs/reference/ingest/processors/community-id.asciidoc

@@ -0,0 +1,99 @@
+[role="xpack"]
+[testenv="basic"]
+[[community-id-processor]]
+=== Community ID processor
+++++
+<titleabbrev>Community ID</titleabbrev>
+++++
+
+Computes the Community ID for network flow data as defined in the
+https://github.com/corelight/community-id-spec[Community ID Specification].
+You can use a community ID to correlate network events related to a single
+flow.
+
+The community ID processor reads network flow data from related
+{ecs-ref}[Elastic Common Schema (ECS)] fields by default. If you use the ECS, no
+configuration is required.
+
+[[community-id-options]]
+.Community ID Options
+[options="header"]
+|======
+| Name               | Required | Default       | Description
+| `source_ip`        | no       | `source.ip`   | Field containing the source IP address.
+| `source_port`      | no       | `source.port` | Field containing the source port.
+| `destination_ip`   | no       | `destination.ip` | Field containing the destination IP address.
+| `destination_port` | no       | `destination.port` | Field containing the destination port.
+| `iana_number`      | no       | `network.iana_number` | Field containing the IANA number.
+| `icmp_type`        | no       | `icmp.type`   | Field containing the ICMP type.
+| `icmp_code`        | no       | `icmp.code`   | Field containing the ICMP code.
+| `transport`        | no       | `network.transport` | Field containing the transport protocol.
+Used only when the `iana_number` field is not present.
+| `target_field`     | no       | `network.community_id` | Output field for the community ID.
+| `seed`             | no       | `0`           | Seed for the community ID hash. Must be between
+0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as
+a staging and production network that use the same addressing scheme.
+| `ignore_missing`   | no       | `true`        | If `true` and any required fields are missing,
+the processor quietly exits without modifying the document.
+
+include::common-options.asciidoc[]
+|======
+
+Here is an example definition of the community ID processor:
+
+[source,js]
+--------------------------------------------------
+{
+  "description" : "...",
+  "processors" : [
+    {
+      "community_id": {
+      }
+    }
+  ]
+}
+--------------------------------------------------
+// NOTCONSOLE
+
+When the above processor executes on the following document:
+
+[source,js]
+--------------------------------------------------
+{
+  "_source": {
+    "source": {
+      "ip": "123.124.125.126",
+      "port": 12345
+    },
+    "destination": {
+      "ip": "55.56.57.58",
+      "port": 80
+    },
+    "network": {
+      "transport": "TCP"
+    }
+  }
+}
+--------------------------------------------------
+// NOTCONSOLE
+
+It produces this result:
+
+[source,js]
+--------------------------------------------------
+"_source" : {
+  "destination" : {
+    "port" : 80,
+    "ip" : "55.56.57.58"
+  },
+  "source" : {
+    "port" : 12345,
+    "ip" : "123.124.125.126"
+  },
+  "network" : {
+    "community_id" : "1:9qr9Z1LViXcNwtLVOHZ3CL8MlyM=",
+    "transport" : "TCP"
+  }
+}
+--------------------------------------------------
+// NOTCONSOLE