elasticsearch/docs/reference/data-streams/lifecycle/tutorial-manage-new-data-stream.asciidoc

[role="xpack"]
[[tutorial-manage-new-data-stream]]
=== Tutorial: Create a data stream with a lifecycle

To create a data stream with a built-in lifecycle, follow these steps:

. <<create-index-template-with-lifecycle>>
. <<create-data-stream-with-lifecycle>>
. <<retrieve-lifecycle-information>>

[discrete]
[[create-index-template-with-lifecycle]]
==== Create an index template

A data stream requires a matching <<index-templates,index template>>. You can configure the data stream lifecycle by
setting the `lifecycle` field in the index template the same as you do for mappings and index settings. You can define an
index template that sets a lifecycle as follows:

* Include the `data_stream` object to enable data streams.

* Define the lifecycle in the template section or include a composable template that defines the lifecycle.

* Use a priority higher than `200` to avoid collisions with built-in templates.
See <<avoid-index-pattern-collisions>>.

You can use the <<indices-put-template,create index template API>>.

[source,console]
--------------------------------------------------
PUT _index_template/my-index-template
{
  "index_patterns": ["my-data-stream*"],
  "data_stream": { },
  "priority": 500,
  "template": {
    "lifecycle": {
      "data_retention": "7d"
    }
  },
  "_meta": {
    "description": "Template with data stream lifecycle"
  }
}
--------------------------------------------------

[discrete]
[[create-data-stream-with-lifecycle]]
==== Create a data stream

You can create a data stream in two ways:

. By manually creating the stream using the <<indices-create-data-stream,create data stream API>>. The stream's name must
still match one of your template's index patterns.
+
[source,console]
--------------------------------------------------
PUT _data_stream/my-data-stream
--------------------------------------------------
// TEST[continued]

. By <<add-documents-to-a-data-stream,indexing requests>> that
target the stream's name. This name must match one of your index template's index patterns.
+
[source,console]
--------------------------------------------------
PUT my-data-stream/_bulk
{ "create":{ } }
{ "@timestamp": "2099-05-06T16:21:15.000Z", "message": "192.0.2.42 - - [06/May/2099:16:21:15 +0000] \"GET /images/bg.jpg HTTP/1.0\" 200 24736" }
{ "create":{ } }
{ "@timestamp": "2099-05-06T16:25:42.000Z", "message": "192.0.2.255 - - [06/May/2099:16:25:42 +0000] \"GET /favicon.ico HTTP/1.0\" 200 3638" }
--------------------------------------------------
// TEST[continued]

[discrete]
[[retrieve-lifecycle-information]]
==== Retrieve lifecycle information

You can use the <<data-streams-get-lifecycle,get data stream lifecycle API>> to see the data stream lifecycle of your data stream and
the <<data-streams-explain-lifecycle,explain data stream lifecycle API>> to see the exact state of each backing index.

[source,console]
--------------------------------------------------
GET _data_stream/my-data-stream/_lifecycle
--------------------------------------------------
// TEST[continued]

The result will look like this:

[source,console-result]
--------------------------------------------------
{
  "data_streams": [
    {
      "name": "my-data-stream",                                     <1>
      "lifecycle": {
        "enabled": true,                                            <2>
        "data_retention": "7d",                                     <3>
        "effective_retention": "7d",                                <4>
        "retention_determined_by": "data_stream_configuration"
      }
    }
  ],
  "global_retention": {}
}
--------------------------------------------------
<1> The name of your data stream.
<2> Shows if the data stream lifecycle is enabled for this data stream.
<3> The retention period of the data indexed in this data stream, as configured by the user.
<4> The retention period that will be applied by the data stream lifecycle. This means that the data in this data stream will
    be kept at least for 7 days. After that {es} can delete it at its own discretion.

If you want to see more information about how the data stream lifecycle is applied on individual backing indices use the
<<data-streams-explain-lifecycle,explain data stream lifecycle API>>:

[source,console]
--------------------------------------------------
GET .ds-my-data-stream-*/_lifecycle/explain
--------------------------------------------------
// TEST[continued]
The result will look like this:

[source,console-result]
--------------------------------------------------
{
  "indices": {
    ".ds-my-data-stream-2023.04.19-000001": {
      "index": ".ds-my-data-stream-2023.04.19-000001",      <1>
      "managed_by_lifecycle": true,                               <2>
      "index_creation_date_millis": 1681918009501,
      "time_since_index_creation": "1.6m",                  <3>
      "lifecycle": {                                        <4>
        "enabled": true,
        "data_retention": "7d"
      }
    }
  }
}
--------------------------------------------------
// TESTRESPONSE[skip:the result is for illustrating purposes only]
<1> The name of the backing index.
<2> If it is managed by the built-in data stream lifecycle.
<3> Time since the index was created.
<4> The lifecycle configuration that is applied on this backing index.

//////////////////////////
[source,console]
--------------------------------------------------
DELETE _data_stream/my-data-stream
DELETE _index_template/my-index-template
--------------------------------------------------
// TEST[continued]

//////////////////////////