Adding upgrade by tier in rolling_upgrade doc (#77491)

* Update rolling_upgrade.asciidoc

As discussed in https://github.com/elastic/elasticsearch/issues/77007#issuecomment-909087934, it was decided that documentation on rolling upgrade should explicitly mention upgrading by tiers.

* Update docs/reference/upgrade/rolling_upgrade.asciidoc

Co-authored-by: Henning Andersen <33268011+henningandersen@users.noreply.github.com>

Co-authored-by: Henning Andersen <33268011+henningandersen@users.noreply.github.com>

docs/reference/upgrade/rolling_upgrade.asciidoc

@@ -13,12 +13,16 @@ into the following two groups and upgrade the groups in this order:
 . Nodes that are not <<master-node,master-eligible>>. You can retrieve a list
 of these nodes with `GET /_nodes/_all,master:false` or by finding all the nodes
 configured with `node.master: false`.
+.. If you are using data tiers, you should
+upgrade the nodes by tier, completing one tier at a time, in this order: frozen, cold, warm and finally hot tier.
+This is to ensure ILM can continue to move between the phases and ensure version
+compatibility.
+.. If you are not using data tiers, you may upgrade the nodes within the group in any order.
+
 
 . Master-eligible nodes, which are the remaining nodes. You can retrieve a list
 of these nodes with `GET /_nodes/master:true`.
 
-You may upgrade the nodes within each of these groups in any order.
-
 Upgrading the nodes in this order ensures that the master-ineligible nodes are
 always running a version at least as new as the master-eligible nodes. Newer
 nodes can always join a cluster with an older master, but older nodes cannot