Fix broken CCR retention lease unfollow test

This commit fixes a broken CCR retention lease unfollow test. The
problem with the test is that the random subset of shards that we picked
to disrupt would not necessarily overlap with the actual shards in
use. We could take a non-empty subset of [0, 3] (e.g., { 2 }) when the
only shard IDs in use were [0, 1]. This commit fixes this by taking into
account the number of shards in use in the test.

With this change, we also take measure to ensure that a successful
branch is tested more frequently than would otherwise be the case. On
that branch, we want to sometimes pretend that the retention lease is
already removed. The randomness here was also sometimes selecting a
subset of shards that did not overlap with the shards actually in use
during the test. While this does not break the test, it is confusing and
reduces the amount of coverage of that branch.

Closes #39185

x-pack/plugin/ccr/src/test/java/org/elasticsearch/xpack/ccr/CcrRetentionLeaseIT.java

@@ -372,8 +372,9 @@ public class CcrRetentionLeaseIT extends CcrIntegTestCase {
     public void testUnfollowRemovesRetentionLeases() throws Exception {
         final String leaderIndex = "leader";
         final String followerIndex = "follower";
+        final int numberOfShards = randomIntBetween(1, 4);
         final String leaderIndexSettings =
-                getIndexSettings(randomIntBetween(1, 4), 0, singletonMap(IndexSettings.INDEX_SOFT_DELETES_SETTING.getKey(), "true"));
+                getIndexSettings(numberOfShards, 0, singletonMap(IndexSettings.INDEX_SOFT_DELETES_SETTING.getKey(), "true"));
         assertAcked(leaderClient().admin().indices().prepareCreate(leaderIndex).setSource(leaderIndexSettings, XContentType.JSON).get());
         final PutFollowAction.Request followRequest = putFollow(leaderIndex, followerIndex);
         followerClient().execute(PutFollowAction.INSTANCE, followRequest).get();
@@ -394,7 +395,9 @@ public class CcrRetentionLeaseIT extends CcrIntegTestCase {
 
         // we will sometimes fake that some of the retention leases are already removed on the leader shard
         final Set<Integer> shardIds =
-                new HashSet<>(randomSubsetOf(randomIntBetween(0, 4), IntStream.range(0, 4).boxed().collect(Collectors.toSet())));
+                new HashSet<>(randomSubsetOf(
+                        randomIntBetween(0, numberOfShards),
+                        IntStream.range(0, numberOfShards).boxed().collect(Collectors.toSet())));
 
         final ClusterStateResponse followerClusterState = followerClient().admin().cluster().prepareState().clear().setNodes(true).get();
         try {
@@ -460,7 +463,6 @@ public class CcrRetentionLeaseIT extends CcrIntegTestCase {
         }
     }
 
-    @AwaitsFix(bugUrl = "https://github.com/elastic/elasticsearch/issues/39185")
     public void testUnfollowFailsToRemoveRetentionLeases() throws Exception {
         final String leaderIndex = "leader";
         final String followerIndex = "follower";
@@ -478,7 +480,9 @@ public class CcrRetentionLeaseIT extends CcrIntegTestCase {
 
         // we will disrupt requests to remove retention leases for these random shards
         final Set<Integer> shardIds =
-                new HashSet<>(randomSubsetOf(randomIntBetween(1, 4), IntStream.range(0, 4).boxed().collect(Collectors.toSet())));
+                new HashSet<>(randomSubsetOf(
+                                randomIntBetween(1, numberOfShards),
+                                IntStream.range(0, numberOfShards).boxed().collect(Collectors.toSet())));
 
         final ClusterStateResponse followerClusterState = followerClient().admin().cluster().prepareState().clear().setNodes(true).get();
         try {