Browse Source

[DOCS] Warn only one date format is added to the field date formats when using dynamic_date_formats (#88915)

* [DOCS] Warn only one date format is added to the field date formats

When using multiple options in `dynamic_date_formats`, only one of the formats of the first document having a date matching one of the date formats provided will be used.

E.g.
```
PUT my-index-000001
{
  "mappings": {
    "dynamic_date_formats": [ "yyyy/MM", "MM/dd/yyyy"]
  }
}

PUT my-index-000001/_doc/1
{
  "create_date": "09/25/2015"
}
```

The generated mappings will be:
```
    "mappings": {
      "dynamic_date_formats": [
        "yyyy/MM",
        "MM/dd/yyyy"
      ],
      "properties": {
        "create_date": {
          "type": "date",
          "format": "MM/dd/yyyy"
        }
      }
    },
```

Indexing a document with `2015/12` would lead to the `format` `"yyyy/MM"` being used for the `create_date`.

This can be misleading especially if the user is using multiple date formats on the same field.
The first document will determine the format of the `date` field being detected.

Maybe we should provide an additional example, such as:
```
PUT my-index-000001
{
  "mappings": {
    "dynamic_date_formats": [ "yyyy/MM||MM/dd/yyyy"]
  }
}
```

My wording is not great, so feel free to amend/edit.

* Update docs/reference/mapping/dynamic/field-mapping.asciidoc

Reword and add code example

* Turned discussion of the two syntaxes into an admonition

* Fix failing tests

Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
Luca Belluccini 3 years ago
parent
commit
2d3bcc483d
1 changed files with 101 additions and 0 deletions
  1. 101 0
      docs/reference/mapping/dynamic/field-mapping.asciidoc

+ 101 - 0
docs/reference/mapping/dynamic/field-mapping.asciidoc

@@ -114,6 +114,107 @@ PUT my-index-000001/_doc/1
 }
 --------------------------------------------------
 
+[NOTE]
+====
+There is a difference between configuring an array of date patterns and
+configuring multiple patterns in a single string separated by `||`. When you
+configure an array of date patterns, the pattern that matches the date in the
+first document with an unmapped date field will determine the mapping of that
+field:
+
+[source,console]
+--------------------------------------------------
+PUT my-index-000001
+{
+  "mappings": {
+    "dynamic_date_formats": [ "yyyy/MM", "MM/dd/yyyy"]
+  }
+}
+
+PUT my-index-000001/_doc/1
+{
+  "create_date": "09/25/2015"
+}
+--------------------------------------------------
+
+The resulting mapping will be:
+
+////
+[source,console]
+----
+GET my-index-000001/_mapping
+----
+//TEST[continued]
+////
+
+[source,console-result]
+--------------------------------------------------
+{
+  "my-index-000001": {
+    "mappings": {
+      "dynamic_date_formats": [
+        "yyyy/MM",
+        "MM/dd/yyyy"
+      ],
+      "properties": {
+        "create_date": {
+          "type": "date",
+          "format": "MM/dd/yyyy"
+        }
+      }
+    }
+  }
+}
+--------------------------------------------------
+
+Configuring multiple patterns in a single string separated by `||` results in a
+mapping that supports any of the date formats. This enables you to index
+documents that use different formats:
+
+[source,console]
+--------------------------------------------------
+PUT my-index-000001
+{
+  "mappings": {
+    "dynamic_date_formats": [ "yyyy/MM||MM/dd/yyyy"]
+  }
+}
+
+PUT my-index-000001/_doc/1
+{
+  "create_date": "09/25/2015"
+}
+--------------------------------------------------
+
+The resulting mapping will be:
+
+////
+[source,console]
+----
+GET my-index-000001/_mapping
+----
+//TEST[continued]
+////
+
+[source,console-result]
+--------------------------------------------------
+{
+  "my-index-000001": {
+    "mappings": {
+      "dynamic_date_formats": [
+        "yyyy/MM||MM/dd/yyyy"
+      ],
+      "properties": {
+        "create_date": {
+          "type": "date",
+          "format": "yyyy/MM||MM/dd/yyyy"
+        }
+      }
+    }
+  }
+}
+--------------------------------------------------
+====
 
 [[numeric-detection]]
 ==== Numeric detection