123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153 |
- [role="xpack"]
- [[tutorial-manage-new-data-stream]]
- === Tutorial: Create a data stream with a lifecycle
- To create a data stream with a built-in lifecycle, follow these steps:
- . <<create-index-template-with-lifecycle>>
- . <<create-data-stream-with-lifecycle>>
- . <<retrieve-lifecycle-information>>
- [discrete]
- [[create-index-template-with-lifecycle]]
- ==== Create an index template
- A data stream requires a matching <<index-templates,index template>>. You can configure the data stream lifecycle by
- setting the `lifecycle` field in the index template the same as you do for mappings and index settings. You can define an
- index template that sets a lifecycle as follows:
- * Include the `data_stream` object to enable data streams.
- * Define the lifecycle in the template section or include a composable template that defines the lifecycle.
- * Use a priority higher than `200` to avoid collisions with built-in templates.
- See <<avoid-index-pattern-collisions>>.
- You can use the <<indices-put-template,create index template API>>.
- [source,console]
- --------------------------------------------------
- PUT _index_template/my-index-template
- {
- "index_patterns": ["my-data-stream*"],
- "data_stream": { },
- "priority": 500,
- "template": {
- "lifecycle": {
- "data_retention": "7d"
- }
- },
- "_meta": {
- "description": "Template with data stream lifecycle"
- }
- }
- --------------------------------------------------
- [discrete]
- [[create-data-stream-with-lifecycle]]
- ==== Create a data stream
- You can create a data stream in two ways:
- . By manually creating the stream using the <<indices-create-data-stream,create data stream API>>. The stream's name must
- still match one of your template's index patterns.
- +
- [source,console]
- --------------------------------------------------
- PUT _data_stream/my-data-stream
- --------------------------------------------------
- // TEST[continued]
- . By <<add-documents-to-a-data-stream,indexing requests>> that
- target the stream's name. This name must match one of your index template's index patterns.
- +
- [source,console]
- --------------------------------------------------
- PUT my-data-stream/_bulk
- { "create":{ } }
- { "@timestamp": "2099-05-06T16:21:15.000Z", "message": "192.0.2.42 - - [06/May/2099:16:21:15 +0000] \"GET /images/bg.jpg HTTP/1.0\" 200 24736" }
- { "create":{ } }
- { "@timestamp": "2099-05-06T16:25:42.000Z", "message": "192.0.2.255 - - [06/May/2099:16:25:42 +0000] \"GET /favicon.ico HTTP/1.0\" 200 3638" }
- --------------------------------------------------
- // TEST[continued]
- [discrete]
- [[retrieve-lifecycle-information]]
- ==== Retrieve lifecycle information
- You can use the <<data-streams-get-lifecycle,get data stream lifecycle API>> to see the data stream lifecycle of your data stream and
- the <<data-streams-explain-lifecycle,explain data stream lifecycle API>> to see the exact state of each backing index.
- [source,console]
- --------------------------------------------------
- GET _data_stream/my-data-stream/_lifecycle
- --------------------------------------------------
- // TEST[continued]
- The result will look like this:
- [source,console-result]
- --------------------------------------------------
- {
- "data_streams": [
- {
- "name": "my-data-stream", <1>
- "lifecycle": {
- "enabled": true, <2>
- "data_retention": "7d", <3>
- "effective_retention": "7d", <4>
- "retention_determined_by": "data_stream_configuration"
- }
- }
- ],
- "global_retention": {}
- }
- --------------------------------------------------
- <1> The name of your data stream.
- <2> Shows if the data stream lifecycle is enabled for this data stream.
- <3> The retention period of the data indexed in this data stream, as configured by the user.
- <4> The retention period that will be applied by the data stream lifecycle. This means that the data in this data stream will
- be kept at least for 7 days. After that {es} can delete it at its own discretion.
- If you want to see more information about how the data stream lifecycle is applied on individual backing indices use the
- <<data-streams-explain-lifecycle,explain data stream lifecycle API>>:
- [source,console]
- --------------------------------------------------
- GET .ds-my-data-stream-*/_lifecycle/explain
- --------------------------------------------------
- // TEST[continued]
- The result will look like this:
- [source,console-result]
- --------------------------------------------------
- {
- "indices": {
- ".ds-my-data-stream-2023.04.19-000001": {
- "index": ".ds-my-data-stream-2023.04.19-000001", <1>
- "managed_by_lifecycle": true, <2>
- "index_creation_date_millis": 1681918009501,
- "time_since_index_creation": "1.6m", <3>
- "lifecycle": { <4>
- "enabled": true,
- "data_retention": "7d"
- }
- }
- }
- }
- --------------------------------------------------
- // TESTRESPONSE[skip:the result is for illustrating purposes only]
- <1> The name of the backing index.
- <2> If it is managed by the built-in data stream lifecycle.
- <3> Time since the index was created.
- <4> The lifecycle configuration that is applied on this backing index.
- //////////////////////////
- [source,console]
- --------------------------------------------------
- DELETE _data_stream/my-data-stream
- DELETE _index_template/my-index-template
- --------------------------------------------------
- // TEST[continued]
- //////////////////////////
|