elasticsearch/docs/reference/data-streams/lifecycle/tutorial-manage-new-data-stream.asciidoc

[role="xpack"]
[[tutorial-manage-new-data-stream]]
=== Tutorial: Create a data stream with a lifecycle

preview::[]

To create a data stream with a built-in lifecycle, follow these steps:

. <<create-index-template-with-lifecycle>>
. <<create-data-stream-with-lifecycle>>
. <<retrieve-lifecycle-information>>

[discrete]
[[create-index-template-with-lifecycle]]
==== Create an index template

A data stream requires a matching <<index-templates,index template>>. You can configure the data stream lifecycle by
setting the `lifecycle` field in the index template the same as you do for mappings and index settings. You can define an
index template that sets a lifecycle as follows:

* Include the `data_stream` object to enable data streams.

* Define the lifecycle in the template section or include a composable template that defines the lifecycle.

* Use a priority higher than `200` to avoid collisions with built-in templates.
See <<avoid-index-pattern-collisions>>.

You can use the <<indices-put-template,create index template API>>.

[source,console]
--------------------------------------------------
PUT _index_template/my-index-template
{
  "index_patterns": ["my-data-stream*"],
  "data_stream": { },
  "priority": 500,
  "template": {
    "lifecycle": {
      "data_retention": "7d"
    }
  },
  "_meta": {
    "description": "Template with data stream lifecycle"
  }
}
--------------------------------------------------

[discrete]
[[create-data-stream-with-lifecycle]]
==== Create a data stream

You can create a data stream in two ways:

. By manually creating the stream using the <<indices-create-data-stream,create data stream API>>. The stream's name must
still match one of your template's index patterns.
+
[source,console]
--------------------------------------------------
PUT _data_stream/my-data-stream
--------------------------------------------------
// TEST[continued]

. By <<add-documents-to-a-data-stream,indexing requests>> that
target the stream's name. This name must match one of your index template's index patterns.
+
[source,console]
--------------------------------------------------
PUT my-data-stream/_bulk
{ "create":{ } }
{ "@timestamp": "2099-05-06T16:21:15.000Z", "message": "192.0.2.42 - - [06/May/2099:16:21:15 +0000] \"GET /images/bg.jpg HTTP/1.0\" 200 24736" }
{ "create":{ } }
{ "@timestamp": "2099-05-06T16:25:42.000Z", "message": "192.0.2.255 - - [06/May/2099:16:25:42 +0000] \"GET /favicon.ico HTTP/1.0\" 200 3638" }
--------------------------------------------------
// TEST[continued]

[discrete]
[[retrieve-lifecycle-information]]
==== Retrieve lifecycle information

You can use the <<data-streams-get-lifecycle,get data stream lifecycle API>> to see the data stream lifecycle of your data stream and
the <<data-streams-explain-lifecycle,explain data stream lifecycle API>> to see the exact state of each backing index.

[source,console]
--------------------------------------------------
GET _data_stream/my-data-stream/_lifecycle
--------------------------------------------------
// TEST[continued]

The result will look like this:

[source,console-result]
--------------------------------------------------
{
  "data_streams": [
    {
      "name": "my-data-stream",                                   <1>
      "lifecycle": {
        "enabled": true,                                          <2>
        "data_retention": "7d",                                   <3>
        "effective_retention": "7d",                              <4>
        "retention_determined_by": "data_stream_configuration"    <5>
      }
    }
  ]
}
--------------------------------------------------
<1> The name of your data stream.
<2> Shows if the data stream lifecycle is enabled for this data stream.
<3> The desired retention period of the data indexed in this data stream, this means that if there are no other limitations
the data for this data stream will be preserved for at least 7 days.
<4> The effective retention, this means that the data in this data stream will
be kept at least for 7 days. After that {es} can delete it at its own discretion.
<5> The configuration that determined the effective retention.

If you want to see more information about how the data stream lifecycle is applied on individual backing indices use the
<<data-streams-explain-lifecycle,explain data stream lifecycle API>>:

[source,console]
--------------------------------------------------
GET .ds-my-data-stream-*/_lifecycle/explain
--------------------------------------------------
// TEST[continued]
The result will look like this:

[source,console-result]
--------------------------------------------------
{
  "indices": {
    ".ds-my-data-stream-2023.04.19-000001": {
      "index": ".ds-my-data-stream-2023.04.19-000001",      <1>
      "managed_by_lifecycle": true,                               <2>
      "index_creation_date_millis": 1681918009501,
      "time_since_index_creation": "1.6m",                  <3>
      "lifecycle": {                                        <4>
        "enabled": true,
        "data_retention": "7d",
        "effective_retention": "7d",
        "retention_determined_by": "data_stream_configuration"
      }
    }
  }
}
--------------------------------------------------
// TESTRESPONSE[skip:the result is for illustrating purposes only]
<1> The name of the backing index.
<2> If it is managed by the built-in data stream lifecycle.
<3> Time since the index was created.
<4> The lifecycle configuration that is applied on this backing index.

//////////////////////////
[source,console]
--------------------------------------------------
DELETE _data_stream/my-data-stream
DELETE _index_template/my-index-template
--------------------------------------------------
// TEST[continued]

//////////////////////////