Browse Source

Implement synthetic source support for range fields (#107081)

* Implement synthetic source support for range fields

This PR adds basic synthetic source support for range fields. There are
following notable properties of synthetic source produced:
* Ranges are always normalized to be inclusive on both ends (this is how
 they are stored).
* Original order of ranges is not preserved.
* Date ranges are always expressed in epoch millis, format is not
preserved.
* IP ranges are always expressed as a range of IPs while it could
have been originally provided as a CIDR.

This PR only implements retrieval of data for source reconstruction from
 doc values.
Oleksandr Kolomiiets 1 year ago
parent
commit
cde894a5ce

+ 5 - 0
docs/changelog/107081.yaml

@@ -0,0 +1,5 @@
+pr: 107081
+summary: Implement synthetic source support for range fields
+area: Mapping
+type: feature
+issues: []

+ 5 - 0
docs/reference/mapping/fields/synthetic-source.asciidoc

@@ -57,6 +57,7 @@ types:
 ** <<ip-synthetic-source,`ip`>>
 ** <<keyword-synthetic-source,`keyword`>>
 ** <<numeric-synthetic-source,`long`>>
+** <<range-synthetic-source,`range` types>>
 ** <<numeric-synthetic-source,`scaled_float`>>
 ** <<numeric-synthetic-source,`short`>>
 ** <<text-synthetic-source,`text`>>
@@ -170,3 +171,7 @@ https://www.rfc-editor.org/rfc/rfc7159.html[JSON RFC] defines objects as
 shouldn't care but without synthetic `_source` the original ordering is
 preserved and some applications may, counter to the spec, do something with
 that ordering.
+
+[[synthetic-source-modifications-ranges]]
+====== Representation of ranges
+Range field vales (e.g. `long_range`) are always represented as inclusive on both sides with bounds adjusted accordingly.  See <<range-synthetic-source-inclusive, examples>>.

+ 206 - 0
docs/reference/mapping/types/range.asciidoc

@@ -220,6 +220,12 @@ The following parameters are accepted by range types:
     Try to convert strings to numbers and truncate fractions for integers.
     Accepts `true` (default) and `false`.
 
+<<doc-values,`doc_values`>>::
+
+    Should the field be stored on disk in a column-stride fashion, so that it
+    can later be used for sorting, aggregations, or scripting? Accepts `true`
+    (default) or `false`.
+
 <<mapping-index,`index`>>::
 
     Should the field be searchable? Accepts `true` (default) and `false`.
@@ -229,3 +235,203 @@ The following parameters are accepted by range types:
     Whether the field value should be stored and retrievable separately from
     the <<mapping-source-field,`_source`>> field. Accepts `true` or `false`
     (default).
+
+[[range-synthetic-source]]
+==== Synthetic `_source`
+
+IMPORTANT: Synthetic `_source` is Generally Available only for TSDB indices
+(indices that have `index.mode` set to `time_series`). For other indices
+synthetic `_source` is in technical preview. Features in technical preview may
+be changed or removed in a future release. Elastic will work to fix
+any issues, but features in technical preview are not subject to the support SLA
+of official GA features.
+
+`range` fields support <<synthetic-source,synthetic `_source`>> in their default
+configuration. Synthetic `_source` cannot be used with <<doc-values,`doc_values`>> disabled.
+
+Synthetic source always sorts values and removes duplicates for all `range` fields except `ip_range` . Ranges are sorted by their lower bound and then by upper bound. For example:
+[source,console,id=synthetic-source-range-sorting-example]
+----
+PUT idx
+{
+  "mappings": {
+    "_source": { "mode": "synthetic" },
+    "properties": {
+      "my_range": { "type": "long_range" }
+    }
+  }
+}
+
+PUT idx/_doc/1
+{
+  "my_range": [
+    {
+        "gte": 200,
+        "lte": 300
+    },
+    {
+        "gte": 1,
+        "lte": 100
+    },
+    {
+        "gte": 200,
+        "lte": 300
+    },
+    {
+        "gte": 200,
+        "lte": 500
+    }
+  ]
+}
+----
+// TEST[s/$/\nGET idx\/_doc\/1?filter_path=_source\n/]
+
+Will become:
+
+[source,console-result]
+----
+{
+  "my_range": [
+    {
+        "gte": 1,
+        "lte": 100
+    },
+    {
+        "gte": 200,
+        "lte": 300
+    },
+    {
+        "gte": 200,
+        "lte": 500
+    }
+  ]
+}
+----
+// TEST[s/^/{"_source":/ s/\n$/}/]
+
+Values of `ip_range` fields are not sorted but original order is not preserved. Duplicate ranges are removed. If `ip_range` field value is provided as a CIDR, it will be represented as a range of IP addresses in synthetic source.
+
+For example:
+[source,console,id=synthetic-source-range-ip-example]
+----
+PUT idx
+{
+  "mappings": {
+    "_source": { "mode": "synthetic" },
+    "properties": {
+      "my_range": { "type": "ip_range" }
+    }
+  }
+}
+
+PUT idx/_doc/1
+{
+  "my_range": [
+    "10.0.0.0/24",
+    {
+      "gte": "10.0.0.0",
+      "lte": "10.0.0.255"
+    }
+  ]
+}
+----
+// TEST[s/$/\nGET idx\/_doc\/1?filter_path=_source\n/]
+
+Will become:
+
+[source,console-result]
+----
+{
+  "my_range": {
+      "gte": "10.0.0.0",
+      "lte": "10.0.0.255"
+    }
+
+}
+----
+// TEST[s/^/{"_source":/ s/\n$/}/]
+
+[[range-synthetic-source-inclusive]]
+Range field vales are always represented as inclusive on both sides with bounds adjusted accordingly. For example:
+[source,console,id=synthetic-source-range-normalization-example]
+----
+PUT idx
+{
+  "mappings": {
+    "_source": { "mode": "synthetic" },
+    "properties": {
+      "my_range": { "type": "long_range" }
+    }
+  }
+}
+
+PUT idx/_doc/1
+{
+  "my_range": {
+    "gt": 200,
+    "lt": 300
+  }
+}
+----
+// TEST[s/$/\nGET idx\/_doc\/1?filter_path=_source\n/]
+
+Will become:
+
+[source,console-result]
+----
+{
+  "my_range": {
+    "gte": 201,
+    "lte": 299
+  }
+}
+----
+// TEST[s/^/{"_source":/ s/\n$/}/]
+
+`date` ranges are formatted using provided `format` or by default using `yyyy-MM-dd'T'HH:mm:ss.SSSZ` format. For example:
+[source,console,id=synthetic-source-range-date-example]
+----
+PUT idx
+{
+  "mappings": {
+    "_source": { "mode": "synthetic" },
+    "properties": {
+      "my_range": { "type": "date_range" }
+    }
+  }
+}
+
+PUT idx/_doc/1
+{
+  "my_range": [
+    {
+      "gte": 1504224000000,
+      "lte": 1504569600000
+    },
+    {
+      "gte": "2017-09-01",
+      "lte": "2017-09-10"
+    }
+  ]
+}
+----
+// TEST[s/$/\nGET idx\/_doc\/1?filter_path=_source\n/]
+
+Will become:
+
+[source,console-result]
+----
+{
+  "my_range": [
+    {
+      "gte": "2017-09-01T00:00:00.000Z",
+      "lte": "2017-09-05T00:00:00.000Z"
+    },
+    {
+      "gte": "2017-09-01T00:00:00.000Z",
+      "lte": "2017-09-10T00:00:00.000Z"
+    }
+  ]
+}
+----
+// TEST[s/^/{"_source":/ s/\n$/}/]

+ 473 - 0
rest-api-spec/src/yamlRestTest/resources/rest-api-spec/test/range/20_synthetic_source.yml

@@ -0,0 +1,473 @@
+setup:
+  - skip:
+      version: " - 8.14.99"
+      reason: synthetic source support added in 8.15
+
+  - do:
+      indices.create:
+        index: synthetic_source_test
+        body:
+          mappings:
+            "_source":
+              "mode": "synthetic"
+            "properties":
+              "integer_range":
+                "type" : "integer_range"
+              "long_range":
+                "type" : "long_range"
+              "float_range":
+                "type" : "float_range"
+              "double_range":
+                "type" : "double_range"
+              "date_range":
+                "type" : "date_range"
+              "ip_range":
+                "type" : "ip_range"
+
+---
+"Integer range":
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "1"
+        body: { "integer_range" : { "gte": 1, "lte": 5 } }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "2"
+        body: { "integer_range" : { "gt": 1, "lte": 3 } }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "3"
+        body: { "integer_range" : [ { "gte": 4, "lt": 5 } ] }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "4"
+        body: { "integer_range" : [ { "gt": 4, "lt": 8 }, { "gt": 4, "lt": 7 } ] }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "5"
+        body: { "integer_range" : null }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "6"
+        body: { "integer_range": { "gte": null, "lte": 10 } }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "7"
+        body: { "integer_range": { "gte": 1 } }
+
+  - do:
+      indices.refresh: {}
+
+  - do:
+      search:
+        index: synthetic_source_test
+  - match: { hits.total.value: 7 }
+  - match:
+      hits.hits.0._source:
+        integer_range:  { "gte": 1, "lte": 5 }
+  - match:
+      hits.hits.1._source:
+        integer_range: { "gte": 2, "lte": 3 }
+  - match:
+      hits.hits.2._source:
+        integer_range: { "gte": 4, "lte": 4 }
+  - match:
+      hits.hits.3._source:
+        integer_range: [ { "gte": 5, "lte": 6 }, { "gte": 5, "lte": 7 } ]
+  - match:
+      hits.hits.4._source: {}
+  - match:
+      hits.hits.5._source:
+        integer_range: { "gte": -2147483648, "lte": 10 }
+  - match:
+      hits.hits.6._source:
+        integer_range: { "gte": 1, "lte": 2147483647 }
+
+---
+"Long range":
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "1"
+        body: { "long_range" : { "gte": 1, "lte": 5 } }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "2"
+        body: { "long_range" : { "gt": 1, "lte": 3 } }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "3"
+        body: { "long_range" : [ { "gte": 4, "lt": 5 } ] }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "4"
+        body: { "long_range" : [ { "gt": 4, "lt": 8 }, { "gt": 4, "lt": 7 } ] }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "5"
+        body: { "long_range" : null }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "6"
+        body: { "long_range": { "gte": null, "lte": 10 } }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "7"
+        body: { "long_range": { "gte": 1 } }
+
+  - do:
+      indices.refresh: {}
+
+  - do:
+      search:
+        index: synthetic_source_test
+  - match: { hits.total.value: 7 }
+  - match:
+      hits.hits.0._source:
+        long_range:  { "gte": 1, "lte": 5 }
+  - match:
+      hits.hits.1._source:
+        long_range: { "gte": 2, "lte": 3 }
+  - match:
+      hits.hits.2._source:
+        long_range: { "gte": 4, "lte": 4 }
+  - match:
+      hits.hits.3._source:
+        long_range: [ { "gte": 5, "lte": 6 }, { "gte": 5, "lte": 7 } ]
+  - match:
+      hits.hits.4._source: {}
+  - match:
+      hits.hits.5._source:
+        long_range: { "gte": -9223372036854775808, "lte": 10 }
+  - match:
+      hits.hits.6._source:
+        long_range: { "gte": 1, "lte": 9223372036854775807 }
+
+---
+"Float range":
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "1"
+        body: { "float_range" : { "gte": 1, "lte": 5 } }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "2"
+        body: { "float_range" : [ { "gte": 4.0, "lte": 5.0 } ] }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "3"
+        body: { "float_range" : [ { "gte": 4, "lte": 8 }, { "gte": 4, "lte": 7 } ] }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "4"
+        body: { "float_range" : null }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "5"
+        body: { "float_range": { "gte": null, "lte": 10 } }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "6"
+        body: { "float_range": { "gte": 1 } }
+
+  - do:
+      indices.refresh: {}
+
+  - do:
+      search:
+        index: synthetic_source_test
+  - match: { hits.total.value: 6 }
+  - match:
+      hits.hits.0._source:
+        float_range:  { "gte": 1.0, "lte": 5.0 }
+  - match:
+      hits.hits.1._source:
+        float_range: { "gte": 4.0, "lte": 5.0 }
+  - match:
+      hits.hits.2._source:
+        float_range: [ { "gte": 4.0, "lte": 7.0 }, { "gte": 4.0, "lte": 8.0 } ]
+  - match:
+      hits.hits.3._source: {}
+  - match:
+      hits.hits.4._source:
+        float_range: { "gte": "-Infinity", "lte": 10.0 }
+  - match:
+      hits.hits.5._source:
+        float_range: { "gte": 1.0, "lte": "Infinity" }
+
+---
+"Double range":
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "1"
+        body: { "double_range" : { "gte": 1, "lte": 5 } }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "2"
+        body: { "double_range" : [ { "gte": 4.0, "lte": 5.0 } ] }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "3"
+        body: { "double_range" : [ { "gte": 4, "lte": 8 }, { "gte": 4, "lte": 7 } ] }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "4"
+        body: { "double_range" : null }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "5"
+        body: { "double_range": { "gte": null, "lte": 10 } }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "6"
+        body: { "double_range": { "gte": 1 } }
+
+  - do:
+      indices.refresh: {}
+
+  - do:
+      search:
+        index: synthetic_source_test
+  - match: { hits.total.value: 6 }
+  - match:
+      hits.hits.0._source:
+        double_range:  { "gte": 1.0, "lte": 5.0 }
+  - match:
+      hits.hits.1._source:
+        double_range: { "gte": 4.0, "lte": 5.0 }
+  - match:
+      hits.hits.2._source:
+        double_range: [ { "gte": 4.0, "lte": 7.0 }, { "gte": 4.0, "lte": 8.0 } ]
+  - match:
+      hits.hits.3._source: {}
+  - match:
+      hits.hits.4._source:
+        double_range: { "gte": "-Infinity", "lte": 10.0 }
+  - match:
+      hits.hits.5._source:
+        double_range: { "gte": 1.0, "lte": "Infinity" }
+
+---
+"IP range":
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "1"
+        body: { "ip_range" : { "gte": "192.168.0.1", "lte": "192.168.0.5" } }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "2"
+        body: { "ip_range" : { "gt": "192.168.0.1", "lte": "192.168.0.3" } }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "3"
+        body: { "ip_range" : [ { "gte": "192.168.0.4", "lt": "192.168.0.5" } ] }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "4"
+        body: { "ip_range" : { "gt": "2001:db8::", "lt": "200a:100::" } }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "5"
+        body: { "ip_range" : "74.125.227.0/25" }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "6"
+        body: { "ip_range" : null }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "7"
+        body: { "ip_range": { "gte": null, "lte": "10.10.10.10" } }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "8"
+        body: { "ip_range": { "gte": "2001:db8::" } }
+
+  - do:
+      indices.refresh: {}
+
+  - do:
+      search:
+        index: synthetic_source_test
+  - match: { hits.total.value: 8 }
+  - match:
+      hits.hits.0._source:
+        ip_range:  { "gte": "192.168.0.1", "lte": "192.168.0.5" }
+  - match:
+      hits.hits.1._source:
+        ip_range: { "gte": "192.168.0.2", "lte": "192.168.0.3" }
+  - match:
+      hits.hits.2._source:
+        ip_range: { "gte": "192.168.0.4", "lte": "192.168.0.4" }
+  - match:
+      hits.hits.3._source:
+        ip_range: { "gte": "2001:db8::1", "lte": "200a:ff:ffff:ffff:ffff:ffff:ffff:ffff" }
+  - match:
+      hits.hits.4._source:
+        ip_range: { "gte": "74.125.227.0", "lte": "74.125.227.127" }
+  - match:
+      hits.hits.5._source: {}
+  - match:
+      hits.hits.6._source:
+        ip_range: { "gte": "0.0.0.0", "lte": "10.10.10.10" }
+  - match:
+      hits.hits.7._source:
+        ip_range: { "gte": "2001:db8::", "lte": "ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff" }
+
+---
+"Date range":
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "1"
+        body: { "date_range" : { "gte": "2017-09-01", "lte": "2017-09-05" } }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "2"
+        body: { "date_range" : { "gt": "2017-09-01", "lte": "2017-09-03" } }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "3"
+        body: { "date_range" : [ { "gte": "2017-09-04", "lt": "2017-09-05" } ] }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "4"
+        body: { "date_range" : [ { "gt": "2017-09-04", "lt": "2017-09-08" }, { "gt": "2017-09-04", "lt": "2017-09-07" } ] }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "5"
+        body: { "date_range" : { "gte": 1504224000000, "lte": 1504569600000 } }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "6"
+        body: { "date_range" : { "gte": "2017-09-01T10:20:30.123Z", "lte": "2017-09-05T03:04:05.789Z" } }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "7"
+        body: { "date_range" : null }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "8"
+        body: { "date_range": { "gte": null, "lte": "2017-09-05" } }
+
+  - do:
+      index:
+        index: synthetic_source_test
+        id: "9"
+        body: { "date_range": { "gte": "2017-09-05" } }
+
+  - do:
+      indices.refresh: {}
+
+  - do:
+      search:
+        index: synthetic_source_test
+  - match: { hits.total.value: 9 }
+  - match:
+      hits.hits.0._source:
+        date_range:  { "gte": "2017-09-01T00:00:00.000Z", "lte": "2017-09-05T00:00:00.000Z" }
+  - match:
+      hits.hits.1._source:
+        date_range: { "gte": "2017-09-01T00:00:00.001Z", "lte": "2017-09-03T00:00:00.000Z" }
+  - match:
+      hits.hits.2._source:
+        date_range: { "gte": "2017-09-04T00:00:00.000Z", "lte": "2017-09-04T23:59:59.999Z" }
+  - match:
+      hits.hits.3._source:
+        date_range: [ { "gte": "2017-09-04T00:00:00.001Z", "lte": "2017-09-06T23:59:59.999Z" }, { "gte": "2017-09-04T00:00:00.001Z", "lte": "2017-09-07T23:59:59.999Z" } ]
+  - match:
+      hits.hits.4._source:
+        date_range: { "gte": "2017-09-01T00:00:00.000Z", "lte": "2017-09-05T00:00:00.000Z" }
+  - match:
+      hits.hits.5._source:
+        date_range: { "gte": "2017-09-01T10:20:30.123Z", "lte": "2017-09-05T03:04:05.789Z" }
+  - match:
+      hits.hits.6._source: {}
+  - match:
+      hits.hits.7._source:
+        date_range: { "gte": "-292275055-05-16T16:47:04.192Z", "lte": "2017-09-05T00:00:00.000Z" }
+  - match:
+      hits.hits.8._source:
+        date_range: { "gte": "2017-09-05T00:00:00.000Z", "lte": "+292278994-08-17T07:12:55.807Z" }
+

+ 33 - 0
server/src/main/java/org/elasticsearch/common/network/InetAddresses.java

@@ -417,4 +417,37 @@ public class InetAddresses {
     public static String toCidrString(InetAddress address, int prefixLength) {
         return new StringBuilder().append(toAddrString(address)).append("/").append(prefixLength).toString();
     }
+
+    /**
+     * Represents a range of IP addresses
+     * @param lowerBound start of the ip range (inclusive)
+     * @param upperBound end of the ip range (inclusive)
+     */
+    public record IpRange(InetAddress lowerBound, InetAddress upperBound) {}
+
+    /**
+     * Parse an IP address and its prefix length using the CIDR notation
+     * into a range of ip addresses corresponding to it.
+     * @param maskedAddress ip address range in a CIDR notation
+     * @throws IllegalArgumentException if the string is not formatted as {@code ip_address/prefix_length}
+     * @throws IllegalArgumentException if the IP address is an IPv6-mapped ipv4 address
+     * @throws IllegalArgumentException if the prefix length is not in 0-32 for IPv4 addresses and 0-128 for IPv6 addresses
+     * @throws NumberFormatException if the prefix length is not an integer
+     */
+    public static IpRange parseIpRangeFromCidr(String maskedAddress) {
+        final Tuple<InetAddress, Integer> cidr = InetAddresses.parseCidr(maskedAddress);
+        // create the lower value by zeroing out the host portion, upper value by filling it with all ones.
+        byte[] lower = cidr.v1().getAddress();
+        byte[] upper = lower.clone();
+        for (int i = cidr.v2(); i < 8 * lower.length; i++) {
+            int m = 1 << 7 - (i & 7);
+            lower[i >> 3] &= (byte) ~m;
+            upper[i >> 3] |= (byte) m;
+        }
+        try {
+            return new IpRange(InetAddress.getByAddress(lower), InetAddress.getByAddress(upper));
+        } catch (UnknownHostException bogus) {
+            throw new AssertionError(bogus);
+        }
+    }
 }

+ 4 - 0
server/src/main/java/org/elasticsearch/index/mapper/BinaryRangeUtil.java

@@ -98,6 +98,10 @@ enum BinaryRangeUtil {
         return decodeRanges(encodedRanges, RangeType.FLOAT, BinaryRangeUtil::decodeFloat);
     }
 
+    static List<RangeFieldMapper.Range> decodeDateRanges(BytesRef encodedRanges) throws IOException {
+        return decodeRanges(encodedRanges, RangeType.DATE, BinaryRangeUtil::decodeLong);
+    }
+
     static List<RangeFieldMapper.Range> decodeRanges(
         BytesRef encodedRanges,
         RangeType rangeType,

+ 123 - 65
server/src/main/java/org/elasticsearch/index/mapper/RangeFieldMapper.java

@@ -26,16 +26,17 @@ import org.elasticsearch.index.fielddata.plain.BinaryIndexFieldData;
 import org.elasticsearch.index.query.SearchExecutionContext;
 import org.elasticsearch.search.DocValueFormat;
 import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
+import org.elasticsearch.xcontent.XContentBuilder;
 import org.elasticsearch.xcontent.XContentParser;
 
 import java.io.IOException;
 import java.net.InetAddress;
-import java.net.UnknownHostException;
 import java.time.ZoneId;
 import java.time.ZoneOffset;
 import java.util.Collections;
 import java.util.HashMap;
 import java.util.HashSet;
+import java.util.List;
 import java.util.Locale;
 import java.util.Map;
 import java.util.Objects;
@@ -380,83 +381,116 @@ public class RangeFieldMapper extends FieldMapper {
 
     @Override
     protected void parseCreateField(DocumentParserContext context) throws IOException {
-        Range range;
         XContentParser parser = context.parser();
-        final XContentParser.Token start = parser.currentToken();
-        if (start == XContentParser.Token.VALUE_NULL) {
+        if (parser.currentToken() == XContentParser.Token.VALUE_NULL) {
             return;
-        } else if (start == XContentParser.Token.START_OBJECT) {
-            RangeFieldType fieldType = fieldType();
-            RangeType rangeType = fieldType.rangeType;
-            String fieldName = null;
-            Object from = rangeType.minValue();
-            Object to = rangeType.maxValue();
-            boolean includeFrom = DEFAULT_INCLUDE_LOWER;
-            boolean includeTo = DEFAULT_INCLUDE_UPPER;
-            XContentParser.Token token;
-            while ((token = parser.nextToken()) != XContentParser.Token.END_OBJECT) {
-                if (token == XContentParser.Token.FIELD_NAME) {
-                    fieldName = parser.currentName();
-                } else {
-                    if (fieldName.equals(GT_FIELD.getPreferredName())) {
-                        includeFrom = false;
-                        if (parser.currentToken() != XContentParser.Token.VALUE_NULL) {
-                            from = rangeType.parseFrom(fieldType, parser, coerce.value(), includeFrom);
-                        }
-                    } else if (fieldName.equals(GTE_FIELD.getPreferredName())) {
-                        includeFrom = true;
-                        if (parser.currentToken() != XContentParser.Token.VALUE_NULL) {
-                            from = rangeType.parseFrom(fieldType, parser, coerce.value(), includeFrom);
-                        }
-                    } else if (fieldName.equals(LT_FIELD.getPreferredName())) {
-                        includeTo = false;
-                        if (parser.currentToken() != XContentParser.Token.VALUE_NULL) {
-                            to = rangeType.parseTo(fieldType, parser, coerce.value(), includeTo);
-                        }
-                    } else if (fieldName.equals(LTE_FIELD.getPreferredName())) {
-                        includeTo = true;
-                        if (parser.currentToken() != XContentParser.Token.VALUE_NULL) {
-                            to = rangeType.parseTo(fieldType, parser, coerce.value(), includeTo);
-                        }
-                    } else {
-                        throw new DocumentParsingException(
-                            parser.getTokenLocation(),
-                            "error parsing field [" + name() + "], with unknown parameter [" + fieldName + "]"
-                        );
-                    }
-                }
-            }
-            range = new Range(rangeType, from, to, includeFrom, includeTo);
-        } else if (fieldType().rangeType == RangeType.IP && start == XContentParser.Token.VALUE_STRING) {
-            range = parseIpRangeFromCidr(parser);
-        } else {
+        }
+
+        Range range = parseRange(parser);
+        context.doc().addAll(fieldType().rangeType.createFields(context, name(), range, index, hasDocValues, store));
+
+        if (hasDocValues == false && (index || store)) {
+            context.addToFieldNames(fieldType().name());
+        }
+    }
+
+    private Range parseRange(XContentParser parser) throws IOException {
+        final XContentParser.Token start = parser.currentToken();
+        if (fieldType().rangeType == RangeType.IP && start == XContentParser.Token.VALUE_STRING) {
+            return parseIpRangeFromCidr(parser);
+        }
+
+        if (start != XContentParser.Token.START_OBJECT) {
             throw new DocumentParsingException(
                 parser.getTokenLocation(),
                 "error parsing field [" + name() + "], expected an object but got " + parser.currentName()
             );
         }
-        context.doc().addAll(fieldType().rangeType.createFields(context, name(), range, index, hasDocValues, store));
 
-        if (hasDocValues == false && (index || store)) {
-            context.addToFieldNames(fieldType().name());
+        RangeFieldType fieldType = fieldType();
+        RangeType rangeType = fieldType.rangeType;
+        String fieldName = null;
+        Object parsedFrom = null;
+        Object parsedTo = null;
+        boolean includeFrom = DEFAULT_INCLUDE_LOWER;
+        boolean includeTo = DEFAULT_INCLUDE_UPPER;
+        XContentParser.Token token;
+        while ((token = parser.nextToken()) != XContentParser.Token.END_OBJECT) {
+            if (token == XContentParser.Token.FIELD_NAME) {
+                fieldName = parser.currentName();
+            } else {
+                if (fieldName.equals(GT_FIELD.getPreferredName())) {
+                    includeFrom = false;
+                    if (parser.currentToken() != XContentParser.Token.VALUE_NULL) {
+                        parsedFrom = rangeType.parseFrom(fieldType, parser, coerce.value(), includeFrom);
+                    }
+                } else if (fieldName.equals(GTE_FIELD.getPreferredName())) {
+                    includeFrom = true;
+                    if (parser.currentToken() != XContentParser.Token.VALUE_NULL) {
+                        parsedFrom = rangeType.parseFrom(fieldType, parser, coerce.value(), includeFrom);
+                    }
+                } else if (fieldName.equals(LT_FIELD.getPreferredName())) {
+                    includeTo = false;
+                    if (parser.currentToken() != XContentParser.Token.VALUE_NULL) {
+                        parsedTo = rangeType.parseTo(fieldType, parser, coerce.value(), includeTo);
+                    }
+                } else if (fieldName.equals(LTE_FIELD.getPreferredName())) {
+                    includeTo = true;
+                    if (parser.currentToken() != XContentParser.Token.VALUE_NULL) {
+                        parsedTo = rangeType.parseTo(fieldType, parser, coerce.value(), includeTo);
+                    }
+                } else {
+                    throw new DocumentParsingException(
+                        parser.getTokenLocation(),
+                        "error parsing field [" + name() + "], with unknown parameter [" + fieldName + "]"
+                    );
+                }
+            }
         }
+        Object from = parsedFrom != null ? parsedFrom : rangeType.defaultFrom(parsedTo);
+        Object to = parsedTo != null ? parsedTo : rangeType.defaultTo(parsedFrom);
+
+        return new Range(rangeType, from, to, includeFrom, includeTo);
     }
 
     private static Range parseIpRangeFromCidr(final XContentParser parser) throws IOException {
-        final Tuple<InetAddress, Integer> cidr = InetAddresses.parseCidr(parser.text());
-        // create the lower value by zeroing out the host portion, upper value by filling it with all ones.
-        byte[] lower = cidr.v1().getAddress();
-        byte[] upper = lower.clone();
-        for (int i = cidr.v2(); i < 8 * lower.length; i++) {
-            int m = 1 << 7 - (i & 7);
-            lower[i >> 3] &= (byte) ~m;
-            upper[i >> 3] |= (byte) m;
-        }
-        try {
-            return new Range(RangeType.IP, InetAddress.getByAddress(lower), InetAddress.getByAddress(upper), true, true);
-        } catch (UnknownHostException bogus) {
-            throw new AssertionError(bogus);
+        final InetAddresses.IpRange range = InetAddresses.parseIpRangeFromCidr(parser.text());
+        return new Range(RangeType.IP, range.lowerBound(), range.upperBound(), true, true);
+    }
+
+    @Override
+    public SourceLoader.SyntheticFieldLoader syntheticFieldLoader() {
+        if (hasDocValues == false) {
+            throw new IllegalArgumentException(
+                "field [" + name() + "] of type [" + typeName() + "] doesn't support synthetic source because it doesn't have doc values"
+            );
+        }
+        if (copyTo.copyToFields().isEmpty() != true) {
+            throw new IllegalArgumentException(
+                "field [" + name() + "] of type [" + typeName() + "] doesn't support synthetic source because it declares copy_to"
+            );
         }
+        return new BinaryDocValuesSyntheticFieldLoader(name()) {
+            @Override
+            protected void writeValue(XContentBuilder b, BytesRef value) throws IOException {
+                List<Range> ranges = type.decodeRanges(value);
+
+                switch (ranges.size()) {
+                    case 0:
+                        return;
+                    case 1:
+                        b.field(simpleName());
+                        ranges.get(0).toXContent(b, fieldType().dateTimeFormatter);
+                        break;
+                    default:
+                        b.startArray(simpleName());
+                        for (var range : ranges) {
+                            range.toXContent(b, fieldType().dateTimeFormatter);
+                        }
+                        b.endArray();
+                }
+            }
+        };
     }
 
     /** Class defining a range */
@@ -516,6 +550,30 @@ public class RangeFieldMapper extends FieldMapper {
         public Object getTo() {
             return to;
         }
+
+        public XContentBuilder toXContent(XContentBuilder builder, DateFormatter dateFormatter) throws IOException {
+            builder.startObject();
+
+            if (includeFrom) {
+                builder.field("gte");
+            } else {
+                builder.field("gt");
+            }
+            Object f = includeFrom || from.equals(type.minValue()) ? from : type.nextDown(from);
+            builder.value(type.formatValue(f, dateFormatter));
+
+            if (includeTo) {
+                builder.field("lte");
+            } else {
+                builder.field("lt");
+            }
+            Object t = includeTo || to.equals(type.maxValue()) ? to : type.nextUp(to);
+            builder.value(type.formatValue(t, dateFormatter));
+
+            builder.endObject();
+
+            return builder;
+        }
     }
 
     static class BinaryRangesDocValuesField extends CustomDocValuesField {

+ 30 - 1
server/src/main/java/org/elasticsearch/index/mapper/RangeType.java

@@ -31,6 +31,7 @@ import org.elasticsearch.lucene.queries.BinaryDocValuesRangeQuery;
 import org.elasticsearch.xcontent.XContentParser;
 
 import java.io.IOException;
+import java.net.Inet4Address;
 import java.net.InetAddress;
 import java.time.Instant;
 import java.time.ZoneId;
@@ -64,6 +65,26 @@ public enum RangeType {
             return included ? address : nextDown(address);
         }
 
+        public Object defaultFrom(Object parsedTo) {
+            if (parsedTo == null) {
+                return minValue();
+            }
+
+            // Make sure that we keep the range inside the same address family.
+            // `minValue()` is always IPv6 so we need to adjust it.
+            return parsedTo instanceof Inet4Address ? InetAddressPoint.decode(new byte[4]) : minValue();
+        }
+
+        public Object defaultTo(Object parsedFrom) {
+            if (parsedFrom == null) {
+                return maxValue();
+            }
+
+            // Make sure that we keep the range inside the same address family.
+            // `maxValue()` is always IPv6 so we need to adjust it.
+            return parsedFrom instanceof Inet4Address ? InetAddressPoint.decode(new byte[] { -1, -1, -1, -1 }) : maxValue();
+        }
+
         @Override
         public InetAddress parseValue(Object value, boolean coerce, @Nullable DateMathParser dateMathParser) {
             if (value instanceof InetAddress) {
@@ -249,7 +270,7 @@ public enum RangeType {
 
         @Override
         public List<RangeFieldMapper.Range> decodeRanges(BytesRef bytes) throws IOException {
-            return LONG.decodeRanges(bytes);
+            return BinaryRangeUtil.decodeDateRanges(bytes);
         }
 
         @Override
@@ -844,6 +865,14 @@ public enum RangeType {
         return included ? value : (Number) nextDown(value);
     }
 
+    public Object defaultFrom(Object parsedTo) {
+        return minValue();
+    }
+
+    public Object defaultTo(Object parsedFrom) {
+        return maxValue();
+    }
+
     public abstract Object minValue();
 
     public abstract Object maxValue();

+ 47 - 4
server/src/test/java/org/elasticsearch/index/mapper/DateRangeFieldMapperTests.java

@@ -8,16 +8,20 @@
 
 package org.elasticsearch.index.mapper;
 
+import org.elasticsearch.common.time.DateFormatter;
+import org.elasticsearch.common.time.DateUtils;
+import org.elasticsearch.xcontent.ToXContent;
 import org.elasticsearch.xcontent.XContentBuilder;
 import org.junit.AssumptionViolatedException;
 
 import java.io.IOException;
+import java.time.Instant;
+import java.util.Map;
+import java.util.stream.Collectors;
 
 import static org.hamcrest.Matchers.containsString;
 
 public class DateRangeFieldMapperTests extends RangeFieldMapperTests {
-
-    private static final String DATE_FORMAT = "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis";
     private static final String FROM_DATE = "2016-10-31";
     private static final String TO_DATE = "2016-11-01 20:00:00";
 
@@ -56,8 +60,47 @@ public class DateRangeFieldMapperTests extends RangeFieldMapperTests {
     }
 
     @Override
-    protected SyntheticSourceSupport syntheticSourceSupport(boolean ignoreMalformed) {
-        throw new AssumptionViolatedException("not supported");
+    @SuppressWarnings("unchecked")
+    protected TestRange<Long> randomRangeForSyntheticSourceTest() {
+        var includeFrom = randomBoolean();
+        Long from = rarely() ? null : randomLongBetween(0, DateUtils.MAX_MILLIS_BEFORE_9999 - 1);
+        var includeTo = randomBoolean();
+        Long to = rarely() ? null : randomLongBetween((from == null ? 0 : from) + 1, DateUtils.MAX_MILLIS_BEFORE_9999);
+
+        return new TestRange<>(rangeType(), from, to, includeFrom, includeTo) {
+            private final DateFormatter inputDateFormatter = DateFormatter.forPattern("yyyy-MM-dd HH:mm:ss.SSS");
+            private final DateFormatter expectedDateFormatter = DateFormatter.forPattern(DATE_FORMAT);
+
+            @Override
+            Object toInput() {
+                var fromKey = includeFrom ? "gte" : "gt";
+                var toKey = includeTo ? "lte" : "lt";
+
+                var fromFormatted = from != null && randomBoolean() ? inputDateFormatter.format(Instant.ofEpochMilli(from)) : from;
+                var toFormatted = to != null && randomBoolean() ? inputDateFormatter.format(Instant.ofEpochMilli(to)) : to;
+
+                return (ToXContent) (builder, params) -> builder.startObject()
+                    .field(fromKey, fromFormatted)
+                    .field(toKey, toFormatted)
+                    .endObject();
+            }
+
+            @Override
+            Object toExpectedSyntheticSource() {
+                Map<String, Object> expectedInMillis = (Map<String, Object>) super.toExpectedSyntheticSource();
+
+                return expectedInMillis.entrySet()
+                    .stream()
+                    .collect(
+                        Collectors.toMap(Map.Entry::getKey, e -> expectedDateFormatter.format(Instant.ofEpochMilli((long) e.getValue())))
+                    );
+            }
+        };
+    }
+
+    @Override
+    protected RangeType rangeType() {
+        return RangeType.DATE;
     }
 
     @Override

+ 68 - 2
server/src/test/java/org/elasticsearch/index/mapper/DoubleRangeFieldMapperTests.java

@@ -8,13 +8,69 @@
 
 package org.elasticsearch.index.mapper;
 
+import org.elasticsearch.core.CheckedConsumer;
+import org.elasticsearch.xcontent.ToXContent;
 import org.elasticsearch.xcontent.XContentBuilder;
 import org.junit.AssumptionViolatedException;
 
 import java.io.IOException;
+import java.util.LinkedHashMap;
+import java.util.List;
+
+import static org.hamcrest.Matchers.equalTo;
 
 public class DoubleRangeFieldMapperTests extends RangeFieldMapperTests {
 
+    public void testSyntheticSourceDefaultValues() throws IOException {
+        // Default range ends for double are negative and positive infinity
+        // and they can not pass `roundTripSyntheticSource` test.
+
+        CheckedConsumer<XContentBuilder, IOException> mapping = b -> {
+            b.startObject("field");
+            minimalMapping(b);
+            b.endObject();
+        };
+
+        var inputValues = List.of(
+            (builder, params) -> builder.startObject().field("gte", (Double) null).field("lte", 10).endObject(),
+            (builder, params) -> builder.startObject().field("lte", 20).endObject(),
+            (builder, params) -> builder.startObject().field("gte", 10).field("lte", (Double) null).endObject(),
+            (builder, params) -> builder.startObject().field("gte", 20).endObject(),
+            (ToXContent) (builder, params) -> builder.startObject().endObject()
+        );
+
+        var expected = List.of(new LinkedHashMap<>() {
+            {
+                put("gte", "-Infinity");
+                put("lte", 10.0);
+            }
+        }, new LinkedHashMap<>() {
+            {
+                put("gte", "-Infinity");
+                put("lte", 20.0);
+            }
+        }, new LinkedHashMap<>() {
+            {
+                put("gte", "-Infinity");
+                put("lte", "Infinity");
+            }
+        }, new LinkedHashMap<>() {
+            {
+                put("gte", 10.0);
+                put("lte", "Infinity");
+            }
+        }, new LinkedHashMap<>() {
+            {
+                put("gte", 20.0);
+                put("lte", "Infinity");
+            }
+        });
+
+        var source = getSourceFor(mapping, inputValues);
+        var actual = source.source().get("field");
+        assertThat(actual, equalTo(expected));
+    }
+
     @Override
     protected XContentBuilder rangeSource(XContentBuilder in) throws IOException {
         return rangeSource(in, "0.5", "2.7");
@@ -41,8 +97,18 @@ public class DoubleRangeFieldMapperTests extends RangeFieldMapperTests {
     }
 
     @Override
-    protected SyntheticSourceSupport syntheticSourceSupport(boolean ignoreMalformed) {
-        throw new AssumptionViolatedException("not supported");
+    protected TestRange<Double> randomRangeForSyntheticSourceTest() {
+        var includeFrom = randomBoolean();
+        Double from = randomDoubleBetween(-Double.MAX_VALUE, Double.MAX_VALUE, true);
+        var includeTo = randomBoolean();
+        Double to = randomDoubleBetween(from, Double.MAX_VALUE, false);
+
+        return new TestRange<>(rangeType(), from, to, includeFrom, includeTo);
+    }
+
+    @Override
+    protected RangeType rangeType() {
+        return RangeType.DOUBLE;
     }
 
     @Override

+ 68 - 2
server/src/test/java/org/elasticsearch/index/mapper/FloatRangeFieldMapperTests.java

@@ -8,13 +8,69 @@
 
 package org.elasticsearch.index.mapper;
 
+import org.elasticsearch.core.CheckedConsumer;
+import org.elasticsearch.xcontent.ToXContent;
 import org.elasticsearch.xcontent.XContentBuilder;
 import org.junit.AssumptionViolatedException;
 
 import java.io.IOException;
+import java.util.LinkedHashMap;
+import java.util.List;
+
+import static org.hamcrest.Matchers.equalTo;
 
 public class FloatRangeFieldMapperTests extends RangeFieldMapperTests {
 
+    public void testSyntheticSourceDefaultValues() throws IOException {
+        // Default range ends for float are negative and positive infinity
+        // and they can not pass `roundTripSyntheticSource` test.
+
+        CheckedConsumer<XContentBuilder, IOException> mapping = b -> {
+            b.startObject("field");
+            minimalMapping(b);
+            b.endObject();
+        };
+
+        var inputValues = List.of(
+            (builder, params) -> builder.startObject().field("gte", (Float) null).field("lte", 10).endObject(),
+            (builder, params) -> builder.startObject().field("lte", 20).endObject(),
+            (builder, params) -> builder.startObject().field("gte", 10).field("lte", (Float) null).endObject(),
+            (builder, params) -> builder.startObject().field("gte", 20).endObject(),
+            (ToXContent) (builder, params) -> builder.startObject().endObject()
+        );
+
+        var expected = List.of(new LinkedHashMap<>() {
+            {
+                put("gte", "-Infinity");
+                put("lte", 10.0);
+            }
+        }, new LinkedHashMap<>() {
+            {
+                put("gte", "-Infinity");
+                put("lte", 20.0);
+            }
+        }, new LinkedHashMap<>() {
+            {
+                put("gte", "-Infinity");
+                put("lte", "Infinity");
+            }
+        }, new LinkedHashMap<>() {
+            {
+                put("gte", 10.0);
+                put("lte", "Infinity");
+            }
+        }, new LinkedHashMap<>() {
+            {
+                put("gte", 20.0);
+                put("lte", "Infinity");
+            }
+        });
+
+        var source = getSourceFor(mapping, inputValues);
+        var actual = source.source().get("field");
+        assertThat(actual, equalTo(expected));
+    }
+
     @Override
     protected XContentBuilder rangeSource(XContentBuilder in) throws IOException {
         return rangeSource(in, "0.5", "2.7");
@@ -41,8 +97,18 @@ public class FloatRangeFieldMapperTests extends RangeFieldMapperTests {
     }
 
     @Override
-    protected SyntheticSourceSupport syntheticSourceSupport(boolean ignoreMalformed) {
-        throw new AssumptionViolatedException("not supported");
+    protected TestRange<Float> randomRangeForSyntheticSourceTest() {
+        var includeFrom = randomBoolean();
+        Float from = (float) randomDoubleBetween(-Float.MAX_VALUE, Float.MAX_VALUE, true);
+        var includeTo = randomBoolean();
+        Float to = (float) randomDoubleBetween(from, Float.MAX_VALUE, false);
+
+        return new TestRange<>(rangeType(), from, to, includeFrom, includeTo);
+    }
+
+    @Override
+    protected RangeType rangeType() {
+        return RangeType.FLOAT;
     }
 
     @Override

+ 19 - 2
server/src/test/java/org/elasticsearch/index/mapper/IntegerRangeFieldMapperTests.java

@@ -35,8 +35,25 @@ public class IntegerRangeFieldMapperTests extends RangeFieldMapperTests {
     }
 
     @Override
-    protected SyntheticSourceSupport syntheticSourceSupport(boolean ignoreMalformed) {
-        throw new AssumptionViolatedException("not supported");
+    protected TestRange<Integer> randomRangeForSyntheticSourceTest() {
+        var includeFrom = randomBoolean();
+        Integer from = randomIntBetween(Integer.MIN_VALUE, Integer.MAX_VALUE - 1);
+        var includeTo = randomBoolean();
+        Integer to = randomIntBetween((from) + 1, Integer.MAX_VALUE);
+
+        if (rarely()) {
+            from = null;
+        }
+        if (rarely()) {
+            to = null;
+        }
+
+        return new TestRange<>(rangeType(), from, to, includeFrom, includeTo);
+    }
+
+    @Override
+    protected RangeType rangeType() {
+        return RangeType.INTEGER;
     }
 
     @Override

+ 150 - 1
server/src/test/java/org/elasticsearch/index/mapper/IpRangeFieldMapperTests.java

@@ -10,15 +10,23 @@ package org.elasticsearch.index.mapper;
 import org.apache.lucene.index.DocValuesType;
 import org.apache.lucene.index.IndexableField;
 import org.elasticsearch.common.network.InetAddresses;
+import org.elasticsearch.core.CheckedConsumer;
+import org.elasticsearch.core.Tuple;
+import org.elasticsearch.xcontent.ToXContent;
 import org.elasticsearch.xcontent.XContentBuilder;
 import org.junit.AssumptionViolatedException;
 
 import java.io.IOException;
+import java.net.InetAddress;
 import java.util.HashMap;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
 import java.util.List;
 import java.util.Map;
 
 import static org.hamcrest.Matchers.containsString;
+import static org.hamcrest.Matchers.equalTo;
+import static org.hamcrest.Matchers.instanceOf;
 
 public class IpRangeFieldMapperTests extends RangeFieldMapperTests {
 
@@ -79,9 +87,150 @@ public class IpRangeFieldMapperTests extends RangeFieldMapperTests {
         }
     }
 
+    @Override
+    public void testNullBounds() throws IOException {
+        DocumentMapper mapper = createDocumentMapper(fieldMapping(b -> {
+            minimalMapping(b);
+            b.field("store", true);
+        }));
+
+        ParsedDocument bothNull = mapper.parse(source(b -> b.startObject("field").nullField("gte").nullField("lte").endObject()));
+        assertThat(storedValue(bothNull), equalTo("[:: : ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff]"));
+
+        ParsedDocument onlyFromIPv4 = mapper.parse(
+            source(b -> b.startObject("field").field("gte", rangeValue()).nullField("lte").endObject())
+        );
+        assertThat(storedValue(onlyFromIPv4), equalTo("[192.168.1.7 : 255.255.255.255]"));
+
+        ParsedDocument onlyToIPv4 = mapper.parse(
+            source(b -> b.startObject("field").nullField("gte").field("lte", rangeValue()).endObject())
+        );
+        assertThat(storedValue(onlyToIPv4), equalTo("[0.0.0.0 : 192.168.1.7]"));
+
+        ParsedDocument onlyFromIPv6 = mapper.parse(
+            source(b -> b.startObject("field").field("gte", "2001:db8::").nullField("lte").endObject())
+        );
+        assertThat(storedValue(onlyFromIPv6), equalTo("[2001:db8:: : ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff]"));
+
+        ParsedDocument onlyToIPv6 = mapper.parse(
+            source(b -> b.startObject("field").nullField("gte").field("lte", "2001:db8::").endObject())
+        );
+        assertThat(storedValue(onlyToIPv6), equalTo("[:: : 2001:db8::]"));
+    }
+
+    @SuppressWarnings("unchecked")
+    public void testValidSyntheticSource() throws IOException {
+        CheckedConsumer<XContentBuilder, IOException> mapping = b -> {
+            b.startObject("field");
+            b.field("type", "ip_range");
+            if (rarely()) {
+                b.field("index", false);
+            }
+            if (rarely()) {
+                b.field("store", false);
+            }
+            b.endObject();
+        };
+
+        var values = randomList(1, 5, this::generateValue);
+        var inputValues = values.stream().map(Tuple::v1).toList();
+        var expectedValues = values.stream().map(Tuple::v2).toList();
+
+        var source = getSourceFor(mapping, inputValues);
+
+        // This is the main reason why we need custom logic.
+        // IP ranges are serialized into binary doc values in unpredictable order
+        // because API uses a set.
+        // So this assert needs to be not sensitive to order and in "reference"
+        // implementation of tests from MapperTestCase it is.
+        var actual = source.source().get("field");
+        if (inputValues.size() == 1) {
+            assertEquals(expectedValues.get(0), actual);
+        } else {
+            assertThat(actual, instanceOf(List.class));
+            assertTrue(((List<Object>) actual).containsAll(new HashSet<>(expectedValues)));
+        }
+    }
+
+    private Tuple<Object, Object> generateValue() {
+        String cidr = randomCidrBlock();
+        InetAddresses.IpRange range = InetAddresses.parseIpRangeFromCidr(cidr);
+
+        var includeFrom = randomBoolean();
+        var includeTo = randomBoolean();
+
+        Object input;
+        // "to" field always comes first.
+        Map<String, Object> output = new LinkedHashMap<>();
+        if (randomBoolean()) {
+            // CIDRs are always inclusive ranges.
+            input = cidr;
+            output.put("gte", InetAddresses.toAddrString(range.lowerBound()));
+            output.put("lte", InetAddresses.toAddrString(range.upperBound()));
+        } else {
+            var fromKey = includeFrom ? "gte" : "gt";
+            var toKey = includeTo ? "lte" : "lt";
+            var from = rarely() ? null : InetAddresses.toAddrString(range.lowerBound());
+            var to = rarely() ? null : InetAddresses.toAddrString(range.upperBound());
+            input = (ToXContent) (builder, params) -> builder.startObject().field(fromKey, from).field(toKey, to).endObject();
+
+            // When ranges are stored, they are always normalized to include both ends.
+            // `includeFrom` and `includeTo` here refers to user input.
+            //
+            // Range values are not properly normalized for default values
+            // which results in off by one error here.
+            // So "gte": null and "gt": null both result in "gte": MIN_VALUE.
+            // This is a bug, see #107282.
+            if (from == null) {
+                output.put("gte", InetAddresses.toAddrString((InetAddress) rangeType().minValue()));
+            } else {
+                var rawFrom = range.lowerBound();
+                var adjustedFrom = includeFrom ? rawFrom : (InetAddress) RangeType.IP.nextUp(rawFrom);
+                output.put("gte", InetAddresses.toAddrString(adjustedFrom));
+            }
+            if (to == null) {
+                output.put("lte", InetAddresses.toAddrString((InetAddress) rangeType().maxValue()));
+            } else {
+                var rawTo = range.upperBound();
+                var adjustedTo = includeTo ? rawTo : (InetAddress) RangeType.IP.nextDown(rawTo);
+                output.put("lte", InetAddresses.toAddrString(adjustedTo));
+            }
+        }
+
+        return Tuple.tuple(input, output);
+    }
+
+    public void testInvalidSyntheticSource() {
+        Exception e = expectThrows(IllegalArgumentException.class, () -> createDocumentMapper(syntheticSourceMapping(b -> {
+            b.startObject("field");
+            b.field("type", "ip_range");
+            b.field("doc_values", false);
+            b.endObject();
+        })));
+        assertThat(
+            e.getMessage(),
+            equalTo("field [field] of type [ip_range] doesn't support synthetic source because it doesn't have doc values")
+        );
+    }
+
     @Override
     protected SyntheticSourceSupport syntheticSourceSupport(boolean ignoreMalformed) {
-        throw new AssumptionViolatedException("not supported");
+        throw new AssumptionViolatedException("custom version of synthetic source tests is implemented");
+    }
+
+    private static String randomCidrBlock() {
+        boolean ipv4 = randomBoolean();
+
+        InetAddress address = randomIp(ipv4);
+        // exclude smallest prefix lengths to avoid empty ranges
+        int prefixLength = ipv4 ? randomIntBetween(0, 30) : randomIntBetween(0, 126);
+
+        return InetAddresses.toCidrString(address, prefixLength);
+    }
+
+    @Override
+    protected RangeType rangeType() {
+        return RangeType.IP;
     }
 
     @Override

+ 19 - 2
server/src/test/java/org/elasticsearch/index/mapper/LongRangeFieldMapperTests.java

@@ -36,8 +36,25 @@ public class LongRangeFieldMapperTests extends RangeFieldMapperTests {
     }
 
     @Override
-    protected SyntheticSourceSupport syntheticSourceSupport(boolean ignoreMalformed) {
-        throw new AssumptionViolatedException("not supported");
+    protected TestRange<Long> randomRangeForSyntheticSourceTest() {
+        var includeFrom = randomBoolean();
+        Long from = randomLongBetween(Long.MIN_VALUE, Long.MAX_VALUE - 1);
+        var includeTo = randomBoolean();
+        Long to = randomLongBetween(from + 1, Long.MAX_VALUE);
+
+        if (rarely()) {
+            from = null;
+        }
+        if (rarely()) {
+            to = null;
+        }
+
+        return new TestRange<>(rangeType(), from, to, includeFrom, includeTo);
+    }
+
+    @Override
+    protected RangeType rangeType() {
+        return RangeType.LONG;
     }
 
     @Override

+ 166 - 2
server/src/test/java/org/elasticsearch/index/mapper/RangeFieldMapperTests.java

@@ -8,14 +8,25 @@
 
 package org.elasticsearch.index.mapper;
 
+import org.apache.lucene.index.DirectoryReader;
 import org.apache.lucene.index.DocValuesType;
 import org.apache.lucene.index.IndexableField;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.tests.index.RandomIndexWriter;
 import org.elasticsearch.core.CheckedConsumer;
+import org.elasticsearch.search.lookup.Source;
+import org.elasticsearch.search.lookup.SourceProvider;
+import org.elasticsearch.xcontent.ToXContent;
 import org.elasticsearch.xcontent.XContentBuilder;
+import org.junit.AssumptionViolatedException;
 
 import java.io.IOException;
 import java.util.Collections;
+import java.util.Comparator;
+import java.util.LinkedHashMap;
 import java.util.List;
+import java.util.Locale;
+import java.util.Map;
 
 import static org.elasticsearch.index.query.RangeQueryBuilder.GTE_FIELD;
 import static org.elasticsearch.index.query.RangeQueryBuilder.GT_FIELD;
@@ -23,10 +34,13 @@ import static org.elasticsearch.index.query.RangeQueryBuilder.LTE_FIELD;
 import static org.elasticsearch.index.query.RangeQueryBuilder.LT_FIELD;
 import static org.hamcrest.Matchers.containsString;
 import static org.hamcrest.Matchers.endsWith;
+import static org.hamcrest.Matchers.equalTo;
 import static org.hamcrest.Matchers.startsWith;
 
 public abstract class RangeFieldMapperTests extends MapperTestCase {
 
+    protected static final String DATE_FORMAT = "uuuu-MM-dd HH:mm:ss.SSS||yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis";
+
     @Override
     protected boolean supportsSearchLookup() {
         return false;
@@ -222,14 +236,14 @@ public abstract class RangeFieldMapperTests extends MapperTestCase {
         }
     }
 
-    private static String storedValue(ParsedDocument doc) {
+    protected static String storedValue(ParsedDocument doc) {
         assertEquals(3, doc.rootDoc().getFields("field").size());
         List<IndexableField> fields = doc.rootDoc().getFields("field");
         IndexableField storedField = fields.get(2);
         return storedField.stringValue();
     }
 
-    public final void testNullBounds() throws IOException {
+    public void testNullBounds() throws IOException {
 
         // null, null => min, max
         assertNullBounds(b -> b.startObject("field").nullField("gte").nullField("lte").endObject(), true, true);
@@ -242,6 +256,156 @@ public abstract class RangeFieldMapperTests extends MapperTestCase {
         assertNullBounds(b -> b.startObject("field").field("gte", val).nullField("lte").endObject(), false, true);
     }
 
+    @Override
+    protected SyntheticSourceSupport syntheticSourceSupport(boolean ignoreMalformed) {
+        assumeTrue("test setup only supports numeric ranges", rangeType().isNumeric());
+
+        return new SyntheticSourceSupport() {
+            @Override
+            public SyntheticSourceExample example(int maxValues) throws IOException {
+                if (randomBoolean()) {
+                    var range = randomRangeForSyntheticSourceTest();
+                    return new SyntheticSourceExample(range.toInput(), range.toExpectedSyntheticSource(), this::mapping);
+                }
+
+                var values = randomList(1, maxValues, () -> randomRangeForSyntheticSourceTest());
+                List<Object> in = values.stream().map(TestRange::toInput).toList();
+                List<Object> outList = values.stream().sorted(Comparator.naturalOrder()).map(TestRange::toExpectedSyntheticSource).toList();
+                Object out = outList.size() == 1 ? outList.get(0) : outList;
+
+                return new SyntheticSourceExample(in, out, this::mapping);
+            }
+
+            private void mapping(XContentBuilder b) throws IOException {
+                b.field("type", rangeType().name);
+                if (rarely()) {
+                    b.field("index", false);
+                }
+                if (rarely()) {
+                    b.field("store", false);
+                }
+                if (rangeType() == RangeType.DATE) {
+                    b.field("format", DATE_FORMAT);
+                }
+            }
+
+            @Override
+            public List<SyntheticSourceInvalidExample> invalidExample() throws IOException {
+                return List.of(
+                    new SyntheticSourceInvalidExample(
+                        equalTo(
+                            String.format(
+                                Locale.ROOT,
+                                "field [field] of type [%s] doesn't support synthetic source because it doesn't have doc values",
+                                rangeType().name
+                            )
+                        ),
+                        b -> b.field("type", rangeType().name).field("doc_values", false)
+                    )
+                );
+            }
+        };
+    }
+
+    /**
+     * Stores range information as if it was provided by user.
+     * Provides an expected value of provided range in synthetic source.
+     * @param <T>
+     */
+    protected class TestRange<T extends Comparable<T>> implements Comparable<TestRange<T>> {
+        private final RangeType type;
+        private final T from;
+        private final T to;
+        private final boolean includeFrom;
+        private final boolean includeTo;
+
+        public TestRange(RangeType type, T from, T to, boolean includeFrom, boolean includeTo) {
+            this.type = type;
+            this.from = from;
+            this.to = to;
+            this.includeFrom = includeFrom;
+            this.includeTo = includeTo;
+        }
+
+        Object toInput() {
+            var fromKey = includeFrom ? "gte" : "gt";
+            var toKey = includeTo ? "lte" : "lt";
+
+            return (ToXContent) (builder, params) -> builder.startObject().field(fromKey, from).field(toKey, to).endObject();
+        }
+
+        Object toExpectedSyntheticSource() {
+            // When ranges are stored, they are always normalized to include both ends.
+            // Also, "to" field always comes first.
+            Map<String, Object> output = new LinkedHashMap<>();
+
+            // Range values are not properly normalized for default values
+            // which results in off by one error here.
+            // So "gte": null and "gt": null both result in "gte": MIN_VALUE.
+            // This is a bug, see #107282.
+            if (from == null) {
+                output.put("gte", rangeType().minValue());
+            } else if (includeFrom) {
+                output.put("gte", from);
+            } else {
+                output.put("gte", type.nextUp(from));
+            }
+
+            if (to == null) {
+                output.put("lte", rangeType().maxValue());
+            } else if (includeTo) {
+                output.put("lte", to);
+            } else {
+                output.put("lte", type.nextDown(to));
+            }
+
+            return output;
+        }
+
+        @Override
+        public int compareTo(TestRange<T> o) {
+            return Comparator.comparing((TestRange<T> r) -> r.from, Comparator.nullsFirst(Comparator.naturalOrder()))
+                .thenComparing((TestRange<T> r) -> r.to)
+                .compare(this, o);
+        }
+    }
+
+    protected TestRange<?> randomRangeForSyntheticSourceTest() {
+        throw new AssumptionViolatedException("Should only be called for specific range types");
+    }
+
+    protected Source getSourceFor(CheckedConsumer<XContentBuilder, IOException> mapping, List<?> inputValues) throws IOException {
+        DocumentMapper mapper = createDocumentMapper(syntheticSourceMapping(mapping));
+
+        CheckedConsumer<XContentBuilder, IOException> input = b -> {
+            b.field("field");
+            if (inputValues.size() == 1) {
+                b.value(inputValues.get(0));
+            } else {
+                b.startArray();
+                for (var range : inputValues) {
+                    b.value(range);
+                }
+                b.endArray();
+            }
+        };
+
+        try (Directory directory = newDirectory()) {
+            RandomIndexWriter iw = new RandomIndexWriter(random(), directory);
+            LuceneDocument doc = mapper.parse(source(input)).rootDoc();
+            iw.addDocument(doc);
+            iw.close();
+            try (DirectoryReader reader = DirectoryReader.open(directory)) {
+                SourceProvider provider = SourceProvider.fromSyntheticSource(mapper.mapping());
+                Source syntheticSource = provider.getSource(getOnlyLeafReader(reader).getContext(), 0);
+
+                return syntheticSource;
+            }
+        }
+    }
+
+    protected abstract RangeType rangeType();
+
     @Override
     protected Object generateRandomInputValue(MappedFieldType ft) {
         // Doc value fetching crashes.