Make order setting required for Realm config (#51195)

The order config must be explicitly specified for each realm.
It must also be unique for each realm. 
This is a breaking change and will begin to take effect in 8.0

Resolves: #37614

docs/reference/migration/migrate_8_0/security.asciidoc

@@ -6,6 +6,29 @@
 //Installation and Upgrade Guide
 
 //tag::notable-breaking-changes[]
+[float]
+==== The realm `order` setting is required
+
+The `xpack.security.authc.realms.{type}.{name}.order` setting is now required and must be
+specified for each explicitly configured realm. Each value must be unique.
+The cluster will fail to start if the requirements are not met.
+
+For example, the following configuration is invalid:
+[source,yaml]
+--------------------------------------------------
+xpack.security.authc.realms.kerberos.kerb1:
+  keytab.path: es.keytab
+  remove_realm_name: false
+--------------------------------------------------
+
+And must be configured as:
+[source,yaml]
+--------------------------------------------------
+xpack.security.authc.realms.kerberos.kerb1:
+  order: 0
+  keytab.path: es.keytab
+  remove_realm_name: false
+--------------------------------------------------
 
 // end::notable-breaking-changes[]
 
@@ -79,8 +102,8 @@ It is now an error to configure any SSL settings for
 For example, the following configuration is invalid:
 [source,yaml]
 --------------------------------------------------
-xpack.security.http.ssl.certificate: elasticsearch.crt 
-xpack.security.http.ssl.key: elasticsearch.key 
+xpack.security.http.ssl.certificate: elasticsearch.crt
+xpack.security.http.ssl.key: elasticsearch.key
 xpack.security.http.ssl.certificate_authorities: [ "corporate-ca.crt" ]
 --------------------------------------------------
 
@@ -88,8 +111,8 @@ And must be configured as either:
 [source,yaml]
 --------------------------------------------------
 xpack.security.http.ssl.enabled: true <1>
-xpack.security.http.ssl.certificate: elasticsearch.crt 
-xpack.security.http.ssl.key: elasticsearch.key 
+xpack.security.http.ssl.certificate: elasticsearch.crt
+xpack.security.http.ssl.key: elasticsearch.key
 xpack.security.http.ssl.certificate_authorities: [ "corporate-ca.crt" ]
 --------------------------------------------------
 <1> or `false`.
@@ -109,5 +132,3 @@ It is now an error to enable SSL for the HTTP (Rest) server without also configu
 a certificate and key through use of the `xpack.security.http.ssl.keystore.path`
 setting or the `xpack.security.http.ssl.certificate` and
 `xpack.security.http.ssl.key` settings.
-
-

docs/reference/settings/security-settings.asciidoc

@@ -189,7 +189,7 @@ namespace in `elasticsearch.yml`. For example:
 xpack.security.authc.realms:
 
     native.realm1: <1>
-        order: 0
+        order: 0 <2>
         ...
 
     ldap.realm2:
@@ -204,6 +204,8 @@ xpack.security.authc.realms:
 <1> Specifies the type of realm (for example, `native`, `ldap`,
 `active_directory`, `pki`, `file`, `kerberos`, `saml`) and the realm name. This
 information is required.
+<2> Specifies priority of a realm in the realm chain. This information
+is required.
 
 The valid settings vary depending on the realm type. For more
 information, see <<setting-up-authentication>>.
@@ -214,8 +216,7 @@ information, see <<setting-up-authentication>>.
 
 `order`::
 The priority of the realm within the realm chain. Realms with a lower order are
-consulted first. Although not required, use of this setting is strongly
-recommended when you configure multiple realms. Defaults to `Integer.MAX_VALUE`.
+consulted first. The value must be unique for each realm. This setting is required.
 
 `enabled`::
 Indicates whether a realm is enabled. You can use this setting to disable a

x-pack/docs/en/security/authentication/configuring-active-directory-realm.asciidoc

@@ -3,10 +3,7 @@ realm and map Active Directory users and groups to roles in the role mapping fil
 
 . Add a realm configuration of type `active_directory` to `elasticsearch.yml`
 under the `xpack.security.authc.realms.active_directory` namespace.
-At a minimum, you must specify the Active Directory `domain_name`.
-If you are configuring multiple realms, you should also 
-explicitly set the `order` attribute to control the order in which the realms 
-are consulted during authentication. 
+At a minimum, you must specify the Active Directory `domain_name` and `order`.
 +
 --
 See <<ref-ad-settings>> for all of the options you can set for an 

x-pack/docs/en/security/authentication/configuring-ldap-realm.asciidoc

@@ -21,11 +21,9 @@ However, multiple bind operations might be needed to find the correct user DN.
 
 .. Add a realm configuration of to `elasticsearch.yml` under the
 `xpack.security.authc.realms.ldap` namespace. At a minimum, you must specify
-the `url` of the LDAP server, and set `user_search.base_dn` to the container DN
-where the users are searched for.
-If you are configuring multiple realms, you should also explicitly set the
-`order` attribute to control the order in which the realms are consulted during 
-authentication. See <<ref-ldap-settings>> for all of the options you can set for 
+the `url` and `order` of the LDAP server, and set `user_search.base_dn` to the
+container DN where the users are searched for.
+See <<ref-ldap-settings>> for all of the options you can set for 
 an `ldap` realm.
 +
 --
@@ -72,10 +70,8 @@ realms you specify are used for authentication. If you also want to use the
 
 .. Add a realm configuration to `elasticsearch.yml` in the
 `xpack.security.authc.realms.ldap` namespace. At a minimum, you must specify
-the `url` of the LDAP server, and specify at least one template with the
-`user_dn_templates` option. If you are configuring multiple realms, you should
-also explicitly set the `order` attribute to control the order in which the
-realms are consulted during authentication.
+the `url` and `order` of the LDAP server, and specify at least one template
+with the `user_dn_templates` option.
 See <<ref-ldap-settings>> for all of the options you can set for an `ldap` realm.
 +
 --
@@ -206,6 +202,7 @@ xpack:
       realms:
         ldap:
           ldap1:
+            order: 0
             metadata: cn
 --------------------------------------------------
 --

x-pack/docs/en/security/authentication/configuring-pki-realm.asciidoc

@@ -21,9 +21,8 @@ clients connect directly to {es}.
 
 . Add a realm configuration for a `pki` realm to `elasticsearch.yml` under the
 `xpack.security.authc.realms.pki` namespace.
-If you are configuring multiple realms, you should 
-explicitly set the `order` attribute. See <<ref-pki-settings>> for all of the 
-options you can set for a `pki` realm.
+You must explicitly set the `order` attribute. See <<ref-pki-settings>> for all
+of the options you can set for a `pki` realm.
 +
 --
 For example, the following snippet shows the most basic `pki` realm configuration:
@@ -61,6 +60,7 @@ xpack:
       realms:
         pki:
           pki1:
+            order: 1
             username_pattern: "EMAILADDRESS=(.*?)(?:,|$)"
 ------------------------------------------------------------
 
@@ -118,6 +118,7 @@ xpack:
       realms:
         pki:
           pki1:
+            order: 1
             truststore:
               path: "pki1_truststore.jks"
 ------------------------------------------------------------

x-pack/docs/en/security/authentication/custom-realm.asciidoc

@@ -89,11 +89,10 @@ under the `xpack.security.authc.realms` namespace.
 You must define your realm within the namespace that matchesto the type defined
 by the extension.
 The options you can set depend on the settings exposed by the custom realm.
-If you are configuring multiple realms, you should also explicitly set the
-`order` attribute to control the order in which the realms are consulted during
-authentication. You should make sure each configured realm has a distinct
-`order` setting. In the event that two or more realms have the same `order`,
-they will be processed in realm `name` order.
+At a minimum, you must explicitly set the `order` attribute to control the
+order in which the realms are consulted during authentication. You must also
+make sure each configured realm has a distinct `order` setting. In the event
+that two or more realms have the same `order`, the node will fail to start.
 +
 IMPORTANT: When you configure realms in `elasticsearch.yml`, only the
 realms you specify are used for authentication. If you also want to use the

x-pack/docs/en/security/authentication/realm-chains.asciidoc

@@ -5,9 +5,9 @@
 <<realms,Realms>> live within a _realm chain_. It is essentially a prioritized
 list of configured realms (typically of various types). Realms are consulted in
 ascending order (that is to say, the realm with the lowest `order` value is
-consulted first). You should make sure each configured realm has a distinct
+consulted first). You must make sure each configured realm has a distinct
 `order` setting. In the event that two or more realms have the same `order`,
-they are processed in `name` order.
+the node will fail to start.
 
 During the authentication process, {stack} {security-features} consult and try
 to authenticate the request one realm at a time. Once one of the realms

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/security/authc/RealmConfig.java

@@ -27,9 +27,15 @@ public class RealmConfig {
         this.identifier = identifier;
         this.settings = settings;
         this.env = env;
+        this.threadContext = threadContext;
         this.enabled = getSetting(RealmSettings.ENABLED_SETTING);
+        if (false == hasSetting(RealmSettings.ORDER_SETTING.apply(type()))) {
+            throw new IllegalArgumentException("'order' is a mandatory parameter for realm config. " +
+                "Found invalid config for realm: '" + identifier.name + "'\n" +
+                "Please see the breaking changes documentation."
+            );
+        }
         this.order = getSetting(RealmSettings.ORDER_SETTING);
-        this.threadContext = threadContext;
     }
 
     public RealmIdentifier identifier() {

x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authc/RealmConfigTests.java

@@ -0,0 +1,51 @@
+/*
+ *
+ *  * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ *  * or more contributor license agreements. Licensed under the Elastic License;
+ *  * you may not use this file except in compliance with the Elastic License.
+ *
+ */
+
+package org.elasticsearch.xpack.core.security.authc;
+
+import org.elasticsearch.common.settings.Settings;
+import org.elasticsearch.common.util.concurrent.ThreadContext;
+import org.elasticsearch.env.Environment;
+import org.elasticsearch.test.ESTestCase;
+import org.junit.Before;
+import org.mockito.Mockito;
+
+import static org.hamcrest.Matchers.containsString;
+
+public class RealmConfigTests extends ESTestCase {
+
+    private RealmConfig.RealmIdentifier realmIdentifier;
+    private Settings globalSettings;
+    private Environment environment;
+    private ThreadContext threadContext;
+
+    @Before
+    public void setUp() throws Exception {
+        realmIdentifier = new RealmConfig.RealmIdentifier(randomAlphaOfLengthBetween(4, 12), randomAlphaOfLengthBetween(4,12));
+        environment = Mockito.mock(Environment.class);
+        globalSettings = Settings.builder().put("path.home", createTempDir()).build();
+        threadContext = new ThreadContext(globalSettings);
+        super.setUp();
+    }
+
+    public void testWillPassWhenOrderSettingIsConfigured() {
+        Settings settings = Settings.builder()
+                .put(globalSettings)
+                .put(RealmSettings.realmSettingPrefix(realmIdentifier) + "order", 0)
+                .build();
+
+        RealmConfig realmConfig = new RealmConfig(realmIdentifier, settings, environment, threadContext);
+        assertEquals(0, realmConfig.order);
+    }
+
+    public void testWillFailWhenOrderSettingIsMissing() {
+        Settings settings = Settings.builder().put(globalSettings).build();
+        var e = expectThrows(IllegalArgumentException.class, () -> new RealmConfig(realmIdentifier, settings, environment, threadContext));
+        assertThat(e.getMessage(), containsString("'order' is a mandatory parameter for realm config"));
+    }
+}

x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/authc/Realms.java

@@ -186,6 +186,7 @@ public class Realms implements Iterable<Realm> {
         List<Realm> realms = new ArrayList<>();
         List<String> kerberosRealmNames = new ArrayList<>();
         Map<String, Set<String>> nameToRealmIdentifier = new HashMap<>();
+        Map<Integer, Set<String>> orderToRealmName = new HashMap<>();
         for (RealmConfig.RealmIdentifier identifier: realmsSettings.keySet()) {
             Realm.Factory factory = factories.get(identifier.getType());
             if (factory == null) {
@@ -218,9 +219,13 @@ public class Realms implements Iterable<Realm> {
             Realm realm = factory.create(config);
             nameToRealmIdentifier.computeIfAbsent(realm.name(), k ->
                 new HashSet<>()).add(RealmSettings.realmSettingPrefix(realm.type()) + realm.name());
+            orderToRealmName.computeIfAbsent(realm.order(), k -> new HashSet<>())
+                .add(realm.name());
             realms.add(realm);
         }
 
+        checkUniqueOrders(orderToRealmName);
+
         if (!realms.isEmpty()) {
             Collections.sort(realms);
         } else {
@@ -305,15 +310,34 @@ public class Realms implements Iterable<Realm> {
     private void addNativeRealms(List<Realm> realms) throws Exception {
         Realm.Factory fileRealm = factories.get(FileRealmSettings.TYPE);
         if (fileRealm != null) {
+            var realmIdentifier = new RealmConfig.RealmIdentifier(FileRealmSettings.TYPE, "default_" + FileRealmSettings.TYPE);
             realms.add(fileRealm.create(new RealmConfig(
-                    new RealmConfig.RealmIdentifier(FileRealmSettings.TYPE, "default_" + FileRealmSettings.TYPE),
-                    settings, env, threadContext)));
+                realmIdentifier,
+                ensureOrderSetting(settings, realmIdentifier, Integer.MIN_VALUE + 1),
+                env, threadContext)));
         }
         Realm.Factory indexRealmFactory = factories.get(NativeRealmSettings.TYPE);
         if (indexRealmFactory != null) {
+            var realmIdentifier = new RealmConfig.RealmIdentifier(NativeRealmSettings.TYPE, "default_" + NativeRealmSettings.TYPE);
             realms.add(indexRealmFactory.create(new RealmConfig(
-                    new RealmConfig.RealmIdentifier(NativeRealmSettings.TYPE, "default_" + NativeRealmSettings.TYPE),
-                    settings, env, threadContext)));
+                realmIdentifier,
+                ensureOrderSetting(settings, realmIdentifier, Integer.MIN_VALUE + 2),
+                env, threadContext)));
+        }
+    }
+
+    private Settings ensureOrderSetting(Settings settings, RealmConfig.RealmIdentifier realmIdentifier, int order) {
+        String orderSettingKey = RealmSettings.realmSettingPrefix(realmIdentifier) + "order";
+        return Settings.builder().put(settings).put(orderSettingKey, order).build();
+    }
+
+    private void checkUniqueOrders(Map<Integer, Set<String>> orderToRealmName) {
+        String duplicateOrders = orderToRealmName.entrySet().stream()
+            .filter(entry -> entry.getValue().size() > 1)
+            .map(entry -> entry.getKey() + ": " + entry.getValue())
+            .collect(Collectors.joining("; "));
+        if (Strings.hasText(duplicateOrders)) {
+            throw new IllegalArgumentException("Found multiple realms configured with the same order: " + duplicateOrders);
         }
     }
 

x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/authc/esnative/ReservedRealm.java

@@ -18,6 +18,7 @@ import org.elasticsearch.threadpool.ThreadPool;
 import org.elasticsearch.xpack.core.XPackSettings;
 import org.elasticsearch.xpack.core.security.authc.AuthenticationResult;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.esnative.ClientReservedRealm;
 import org.elasticsearch.xpack.core.security.authc.support.Hasher;
 import org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken;
@@ -63,7 +64,11 @@ public class ReservedRealm extends CachingUsernamePasswordRealm {
 
     public ReservedRealm(Environment env, Settings settings, NativeUsersStore nativeUsersStore, AnonymousUser anonymousUser,
                          SecurityIndexManager securityIndex, ThreadPool threadPool) {
-        super(new RealmConfig(new RealmConfig.RealmIdentifier(TYPE, TYPE), settings, env, threadPool.getThreadContext()), threadPool);
+        super(new RealmConfig(new RealmConfig.RealmIdentifier(TYPE, TYPE),
+            Settings.builder()
+                .put(settings)
+                .put(RealmSettings.realmSettingPrefix(new RealmConfig.RealmIdentifier(TYPE, TYPE)) + "order", Integer.MIN_VALUE)
+                .build(), env, threadPool.getThreadContext()), threadPool);
         this.nativeUsersStore = nativeUsersStore;
         this.realmEnabled = XPackSettings.RESERVED_REALM_ENABLED_SETTING.get(settings);
         this.anonymousUser = anonymousUser;

x-pack/plugin/security/src/test/java/org/elasticsearch/test/SecuritySettingsSource.java

@@ -185,7 +185,7 @@ public class SecuritySettingsSource extends NodeConfigurationSource {
     protected SecureString nodeClientPassword() {
         return new SecureString(TEST_PASSWORD.toCharArray());
     }
-    
+
     public static void addSSLSettingsForNodePEMFiles(Settings.Builder builder, String prefix, boolean hostnameVerificationEnabled) {
         addSSLSettingsForPEMFiles(builder, prefix,
             "/org/elasticsearch/xpack/security/transport/ssl/certs/simple/testnode.pem", "testnode",

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/action/oidc/TransportOpenIdConnectLogoutActionTests.java

@@ -44,6 +44,7 @@ import org.elasticsearch.xpack.core.security.action.oidc.OpenIdConnectLogoutRequ
 import org.elasticsearch.xpack.core.security.action.oidc.OpenIdConnectLogoutResponse;
 import org.elasticsearch.xpack.core.security.authc.Authentication;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.oidc.OpenIdConnectRealmSettings;
 import org.elasticsearch.xpack.core.security.user.User;
 import org.elasticsearch.xpack.core.ssl.SSLService;
@@ -88,9 +89,11 @@ public class TransportOpenIdConnectLogoutActionTests extends OpenIdConnectTestCa
 
     @Before
     public void setup() throws Exception {
+        final RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("oidc", REALM_NAME);
         final Settings settings = getBasicRealmSettings()
             .put(XPackSettings.TOKEN_SERVICE_ENABLED_SETTING.getKey(), true)
             .put("path.home", createTempDir())
+            .put(RealmSettings.getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0)
             .build();
         final Settings sslSettings = Settings.builder()
             .put("xpack.security.authc.realms.oidc.oidc-realm.ssl.verification_mode", "certificate")
@@ -179,8 +182,6 @@ public class TransportOpenIdConnectLogoutActionTests extends OpenIdConnectTestCa
 
         final Environment env = TestEnvironment.newEnvironment(settings);
 
-        final RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("oidc", REALM_NAME);
-
         final RealmConfig realmConfig = new RealmConfig(realmIdentifier, settings, env, threadContext);
         oidcRealm = new OpenIdConnectRealm(realmConfig, new SSLService(TestEnvironment.newEnvironment(sslSettings)),
             mock(UserRoleMapper.class), mock(ResourceWatcherService.class));

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/action/saml/TransportSamlInvalidateSessionActionTests.java

@@ -63,6 +63,7 @@ import org.elasticsearch.xpack.core.security.authc.Authentication;
 import org.elasticsearch.xpack.core.security.authc.Authentication.RealmRef;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig.RealmIdentifier;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.esnative.NativeRealmSettings;
 import org.elasticsearch.xpack.core.security.authc.saml.SamlRealmSettings;
 import org.elasticsearch.xpack.core.security.user.User;
@@ -121,6 +122,7 @@ public class TransportSamlInvalidateSessionActionTests extends SamlTestCase {
 
     @Before
     public void setup() throws Exception {
+        final RealmIdentifier realmId = new RealmIdentifier("saml", REALM_NAME);
         final Path metadata = PathUtils.get(SamlRealm.class.getResource("idp1.xml").toURI());
         final Settings settings = Settings.builder()
             .put(XPackSettings.TOKEN_SERVICE_ENABLED_SETTING.getKey(), true)
@@ -131,6 +133,7 @@ public class TransportSamlInvalidateSessionActionTests extends SamlTestCase {
             .put(getFullSettingKey(REALM_NAME, SamlRealmSettings.SP_ACS), SamlRealmTestHelper.SP_ACS_URL)
             .put(getFullSettingKey(REALM_NAME, SamlRealmSettings.SP_LOGOUT), SamlRealmTestHelper.SP_LOGOUT_URL)
             .put(getFullSettingKey(REALM_NAME, SamlRealmSettings.PRINCIPAL_ATTRIBUTE.getAttribute()), "uid")
+            .put(getFullSettingKey(realmId, RealmSettings.ORDER_SETTING), 0)
             .build();
 
         final ThreadContext threadContext = new ThreadContext(settings);
@@ -215,7 +218,6 @@ public class TransportSamlInvalidateSessionActionTests extends SamlTestCase {
 
         final Environment env = TestEnvironment.newEnvironment(settings);
 
-        final RealmIdentifier realmId = new RealmIdentifier("saml", REALM_NAME);
         final RealmConfig realmConfig = new RealmConfig(
                 realmId,
             settings,

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/action/saml/TransportSamlLogoutActionTests.java

@@ -52,6 +52,7 @@ import org.elasticsearch.xpack.core.security.action.saml.SamlLogoutResponse;
 import org.elasticsearch.xpack.core.security.authc.Authentication;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig.RealmIdentifier;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.saml.SamlRealmSettings;
 import org.elasticsearch.xpack.core.security.user.User;
 import org.elasticsearch.xpack.core.ssl.SSLService;
@@ -103,6 +104,7 @@ public class TransportSamlLogoutActionTests extends SamlTestCase {
 
     @Before
     public void setup() throws Exception {
+        final RealmIdentifier realmIdentifier = new RealmIdentifier("saml", REALM_NAME);
         final Path metadata = PathUtils.get(SamlRealm.class.getResource("idp1.xml").toURI());
         final Settings settings = Settings.builder()
             .put(XPackSettings.TOKEN_SERVICE_ENABLED_SETTING.getKey(), true)
@@ -112,6 +114,7 @@ public class TransportSamlLogoutActionTests extends SamlTestCase {
             .put(getFullSettingKey(REALM_NAME, SamlRealmSettings.SP_ENTITY_ID), SP_URL)
             .put(getFullSettingKey(REALM_NAME, SamlRealmSettings.SP_ACS), SP_URL)
             .put(getFullSettingKey(REALM_NAME, SamlRealmSettings.PRINCIPAL_ATTRIBUTE.getAttribute()), "uid")
+            .put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0)
             .build();
 
         final ThreadContext threadContext = new ThreadContext(settings);
@@ -212,8 +215,6 @@ public class TransportSamlLogoutActionTests extends SamlTestCase {
 
         final Environment env = TestEnvironment.newEnvironment(settings);
 
-        final RealmIdentifier realmIdentifier = new RealmIdentifier("saml", REALM_NAME);
-
         final RealmConfig realmConfig = new RealmConfig(realmIdentifier, settings, env, threadContext);
         samlRealm = SamlRealm.create(realmConfig, mock(SSLService.class), mock(ResourceWatcherService.class), mock(UserRoleMapper.class));
         when(realms.realm(realmConfig.name())).thenReturn(samlRealm);

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/InternalRealmsTests.java

@@ -14,6 +14,7 @@ import org.elasticsearch.threadpool.ThreadPool;
 import org.elasticsearch.watcher.ResourceWatcherService;
 import org.elasticsearch.xpack.core.security.authc.Realm;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.esnative.NativeRealmSettings;
 import org.elasticsearch.xpack.core.security.authc.file.FileRealmSettings;
 import org.elasticsearch.xpack.core.security.authc.kerberos.KerberosRealmSettings;
@@ -47,8 +48,9 @@ public class InternalRealmsTests extends ESTestCase {
         assertThat(factories, hasEntry(is(NativeRealmSettings.TYPE), any(Realm.Factory.class)));
         verifyZeroInteractions(securityIndex);
 
-        Settings settings = Settings.builder().put("path.home", createTempDir()).build();
         final RealmConfig.RealmIdentifier realmId = new RealmConfig.RealmIdentifier(NativeRealmSettings.TYPE, "test");
+        Settings settings = Settings.builder().put("path.home", createTempDir())
+            .put(RealmSettings.getFullSettingKey(realmId, RealmSettings.ORDER_SETTING), 0).build();
         final Environment env = TestEnvironment.newEnvironment(settings);
         final ThreadContext threadContext = new ThreadContext(settings);
         factories.get(NativeRealmSettings.TYPE).create(new RealmConfig(realmId, settings, env, threadContext));

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/RealmsTests.java

@@ -65,7 +65,7 @@ public class RealmsTests extends ESTestCase {
         factories.put(FileRealmSettings.TYPE, config -> new DummyRealm(FileRealmSettings.TYPE, config));
         factories.put(NativeRealmSettings.TYPE, config -> new DummyRealm(NativeRealmSettings.TYPE, config));
         factories.put(KerberosRealmSettings.TYPE, config -> new DummyRealm(KerberosRealmSettings.TYPE, config));
-        randomRealmTypesCount = randomIntBetween(1, 5);
+        randomRealmTypesCount = randomIntBetween(2, 5);
         for (int i = 0; i < randomRealmTypesCount; i++) {
             String name = "type_" + i;
             factories.put(name, config -> new DummyRealm(name, config));
@@ -134,26 +134,10 @@ public class RealmsTests extends ESTestCase {
         }
         Settings settings = builder.build();
         Environment env = TestEnvironment.newEnvironment(settings);
-        Realms realms = new Realms(settings, env, factories, licenseState, threadContext, reservedRealm);
-
-        Iterator<Realm> iterator = realms.iterator();
-        assertThat(iterator.hasNext(), is(true));
-        Realm realm = iterator.next();
-        assertThat(realm, is(reservedRealm));
-
-        // As order is same for all realms, it should fall back secondary comparison on name
-        // Verify that realms are iterated in order based on name
-        Iterator<String> expectedSortedOrderNames = nameToRealmId.keySet().iterator();
-        while (iterator.hasNext()) {
-            realm = iterator.next();
-            String expectedRealmName = expectedSortedOrderNames.next();
-            assertThat(realm.order(), equalTo(1));
-            assertThat(realm.type(), equalTo("type_" + nameToRealmId.get(expectedRealmName)));
-            assertThat(realm.name(), equalTo(expectedRealmName));
-        }
-
-        assertThat(realms.getUnlicensedRealms(), empty());
-        assertThat(realms.getUnlicensedRealms(), sameInstance(realms.getUnlicensedRealms()));
+        IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () ->{
+            new Realms(settings, env, factories, licenseState, threadContext, reservedRealm);
+        });
+        assertThat(e.getMessage(), containsString("Found multiple realms configured with the same order"));
     }
 
     public void testWithSettingsWithMultipleInternalRealmsOfSameType() throws Exception {

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/esnative/NativeRealmTests.java

@@ -13,6 +13,7 @@ import org.elasticsearch.env.TestEnvironment;
 import org.elasticsearch.test.ESTestCase;
 import org.elasticsearch.threadpool.ThreadPool;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.index.RestrictedIndicesNames;
 import org.elasticsearch.xpack.security.support.SecurityIndexManager;
 
@@ -38,8 +39,9 @@ public class NativeRealmTests extends ESTestCase {
         when(threadPool.getThreadContext()).thenReturn(threadContext);
         final AtomicInteger numInvalidation = new AtomicInteger(0);
         int expectedInvalidation = 0;
-        Settings settings = Settings.builder().put("path.home", createTempDir()).build();
         RealmConfig.RealmIdentifier realmId = new RealmConfig.RealmIdentifier("native", "native");
+        Settings settings = Settings.builder().put("path.home", createTempDir())
+            .put(RealmSettings.realmSettingPrefix(realmId) + "order", 0).build();
         RealmConfig config = new RealmConfig(realmId, settings, TestEnvironment.newEnvironment(settings), new ThreadContext(settings));
         final NativeRealm nativeRealm = new NativeRealm(config, mock(NativeUsersStore.class), threadPool) {
             @Override

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/file/FileRealmTests.java

@@ -61,7 +61,8 @@ public class FileRealmTests extends ESTestCase {
         userPasswdStore = mock(FileUserPasswdStore.class);
         userRolesStore = mock(FileUserRolesStore.class);
         globalSettings = Settings.builder().put("path.home", createTempDir()).put("xpack.security.authc.password_hashing.algorithm",
-            randomFrom("bcrypt9", "pbkdf2")).build();
+            randomFrom("bcrypt9", "pbkdf2")).
+            put(RealmSettings.realmSettingPrefix(REALM_IDENTIFIER) + "order", 0).build();
         threadPool = mock(ThreadPool.class);
         threadContext = new ThreadContext(globalSettings);
         when(threadPool.getThreadContext()).thenReturn(threadContext);
@@ -243,8 +244,8 @@ public class FileRealmTests extends ESTestCase {
 
         final int order = randomIntBetween(0, 10);
         Settings settings = Settings.builder()
-            .put(RealmSettings.realmSettingPrefix(REALM_IDENTIFIER) + "order", order)
             .put(globalSettings)
+            .put(RealmSettings.realmSettingPrefix(REALM_IDENTIFIER) + "order", order)
             .build();
 
         RealmConfig config = getRealmConfig(settings);

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/file/FileUserPasswdStoreTests.java

@@ -18,6 +18,7 @@ import org.elasticsearch.watcher.ResourceWatcherService;
 import org.elasticsearch.xpack.core.security.audit.logfile.CapturingLogger;
 import org.elasticsearch.xpack.core.security.authc.AuthenticationResult;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.support.Hasher;
 import org.elasticsearch.xpack.core.security.user.User;
 import org.junit.After;
@@ -134,7 +135,9 @@ public class FileUserPasswdStoreTests extends ESTestCase {
 
     private RealmConfig getRealmConfig() {
         final RealmConfig.RealmIdentifier identifier = new RealmConfig.RealmIdentifier("file", "file-test");
-        return new RealmConfig(identifier, settings, env, threadPool.getThreadContext());
+        return new RealmConfig(identifier,
+            Settings.builder().put(settings).put(RealmSettings.getFullSettingKey(identifier, RealmSettings.ORDER_SETTING), 0).build(),
+            env, threadPool.getThreadContext());
     }
 
     public void testStore_AutoReload_WithParseFailures() throws Exception {

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/file/FileUserRolesStoreTests.java

@@ -19,6 +19,7 @@ import org.elasticsearch.watcher.ResourceWatcherService;
 import org.elasticsearch.xpack.core.XPackSettings;
 import org.elasticsearch.xpack.core.security.audit.logfile.CapturingLogger;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.junit.After;
 import org.junit.Before;
 
@@ -75,7 +76,9 @@ public class FileUserRolesStoreTests extends ESTestCase {
         Files.write(file, lines, StandardCharsets.UTF_16);
 
         RealmConfig.RealmIdentifier realmId = new RealmConfig.RealmIdentifier("file", "file-test");
-        RealmConfig config = new RealmConfig(realmId, settings, env, new ThreadContext(Settings.EMPTY));
+        RealmConfig config = new RealmConfig(realmId,
+            Settings.builder().put(settings).put(RealmSettings.getFullSettingKey(realmId, RealmSettings.ORDER_SETTING), 0).build(),
+            env, new ThreadContext(Settings.EMPTY));
         ResourceWatcherService watcherService = new ResourceWatcherService(settings, threadPool);
         FileUserRolesStore store = new FileUserRolesStore(config, watcherService);
         assertThat(store.entriesCount(), is(0));
@@ -88,7 +91,9 @@ public class FileUserRolesStoreTests extends ESTestCase {
 
 
         final RealmConfig.RealmIdentifier realmId = new RealmConfig.RealmIdentifier("file", "file-test");
-        RealmConfig config = new RealmConfig(realmId, settings, env, new ThreadContext(Settings.EMPTY));
+        RealmConfig config = new RealmConfig(realmId,
+            Settings.builder().put(settings).put(RealmSettings.getFullSettingKey(realmId, RealmSettings.ORDER_SETTING), 0).build(),
+            env, new ThreadContext(Settings.EMPTY));
         ResourceWatcherService watcherService = new ResourceWatcherService(settings, threadPool);
         final CountDownLatch latch = new CountDownLatch(1);
 
@@ -134,7 +139,9 @@ public class FileUserRolesStoreTests extends ESTestCase {
         Files.copy(users, tmp, StandardCopyOption.REPLACE_EXISTING);
 
         final RealmConfig.RealmIdentifier realmId = new RealmConfig.RealmIdentifier("file", "file-test");
-        RealmConfig config = new RealmConfig(realmId, settings, env, new ThreadContext(Settings.EMPTY));
+        RealmConfig config = new RealmConfig(realmId,
+            Settings.builder().put(settings).put(RealmSettings.getFullSettingKey(realmId, RealmSettings.ORDER_SETTING), 0).build(),
+            env, new ThreadContext(Settings.EMPTY));
         ResourceWatcherService watcherService = new ResourceWatcherService(settings, threadPool);
         final CountDownLatch latch = new CountDownLatch(1);
 
@@ -217,13 +224,14 @@ public class FileUserRolesStoreTests extends ESTestCase {
             threadPool = new TestThreadPool("test");
             Path usersRoles = writeUsersRoles("role1:admin");
 
+            final RealmConfig.RealmIdentifier realmId = new RealmConfig.RealmIdentifier("file", "file-test");
             Settings settings = Settings.builder()
                     .put(XPackSettings.WATCHER_ENABLED.getKey(), "false")
                     .put("path.home", createTempDir())
+                    .put(RealmSettings.getFullSettingKey(realmId, RealmSettings.ORDER_SETTING), 0)
                     .build();
 
             Environment env = TestEnvironment.newEnvironment(settings);
-            final RealmConfig.RealmIdentifier realmId = new RealmConfig.RealmIdentifier("file", "file-test");
             RealmConfig config = new RealmConfig(realmId, settings, env, new ThreadContext(Settings.EMPTY));
             ResourceWatcherService watcherService = new ResourceWatcherService(settings, threadPool);
             FileUserRolesStore store = new FileUserRolesStore(config, watcherService);

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/kerberos/KerberosRealmAuthenticateFailedTests.java

@@ -16,6 +16,7 @@ import org.elasticsearch.common.util.concurrent.ThreadContext;
 import org.elasticsearch.env.TestEnvironment;
 import org.elasticsearch.xpack.core.security.authc.AuthenticationResult;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.kerberos.KerberosRealmSettings;
 import org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken;
 import org.elasticsearch.xpack.core.security.user.User;
@@ -121,8 +122,11 @@ public class KerberosRealmAuthenticateFailedTests extends KerberosRealmTestCase
 
     public void testDelegatedAuthorizationFailedToResolve() throws Exception {
         final String username = randomPrincipalName();
-        final MockLookupRealm otherRealm = new MockLookupRealm(new RealmConfig(new RealmConfig.RealmIdentifier("mock", "other_realm"),
-            globalSettings, TestEnvironment.newEnvironment(globalSettings), new ThreadContext(globalSettings)));
+        RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("mock", "other_realm");
+        final MockLookupRealm otherRealm = new MockLookupRealm(new RealmConfig(realmIdentifier,
+            Settings.builder().put(globalSettings)
+                .put(RealmSettings.getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build(),
+            TestEnvironment.newEnvironment(globalSettings), new ThreadContext(globalSettings)));
         final User lookupUser = new User(randomAlphaOfLength(5));
         otherRealm.registerUser(lookupUser);
 

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/kerberos/KerberosRealmTestCase.java

@@ -137,6 +137,7 @@ public abstract class KerberosRealmTestCase extends ESTestCase {
         return Settings.builder().put(realmSettings)
             .normalizePrefix(RealmSettings.realmSettingPrefix(identifier))
             .put(globalSettings)
+            .put(RealmSettings.getFullSettingKey(identifier, RealmSettings.ORDER_SETTING), 0)
             .build();
     }
 
@@ -266,6 +267,7 @@ public abstract class KerberosRealmTestCase extends ESTestCase {
             .put(RealmSettings.getFullSettingKey(realmName, KerberosRealmSettings.CACHE_TTL_SETTING), cacheTTL)
             .put(RealmSettings.getFullSettingKey(realmName, KerberosRealmSettings.SETTING_KRB_DEBUG_ENABLE), enableDebugging)
             .put(RealmSettings.getFullSettingKey(realmName, KerberosRealmSettings.SETTING_REMOVE_REALM_NAME), removeRealmName)
+            .put(RealmSettings.getFullSettingKey(realmName, RealmSettings.ORDER_SETTING.apply(KerberosRealmSettings.TYPE)), 0)
             .put(globalSettings);
         return builder.build();
     }

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/kerberos/KerberosRealmTests.java

@@ -18,6 +18,7 @@ import org.elasticsearch.env.TestEnvironment;
 import org.elasticsearch.rest.RestStatus;
 import org.elasticsearch.xpack.core.security.authc.AuthenticationResult;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.kerberos.KerberosRealmSettings;
 import org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken;
 import org.elasticsearch.xpack.core.security.user.User;
@@ -174,8 +175,12 @@ public class KerberosRealmTests extends KerberosRealmTestCase {
     public void testDelegatedAuthorization() throws Exception {
         final String username = randomPrincipalName();
         final String expectedUsername = maybeRemoveRealmName(username);
-        final MockLookupRealm otherRealm = spy(new MockLookupRealm(new RealmConfig(new RealmConfig.RealmIdentifier("mock", "other_realm"),
-            globalSettings, TestEnvironment.newEnvironment(globalSettings), new ThreadContext(globalSettings))));
+        RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("mock", "other_realm");
+        final MockLookupRealm otherRealm = spy(new MockLookupRealm(new RealmConfig(
+            realmIdentifier,
+            Settings.builder().put(globalSettings)
+                .put(RealmSettings.getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build(),
+            TestEnvironment.newEnvironment(globalSettings), new ThreadContext(globalSettings))));
         final User lookupUser = new User(expectedUsername, new String[] { "admin-role" }, expectedUsername,
                 expectedUsername + "@example.com", Collections.singletonMap("k1", "v1"), true);
         otherRealm.registerUser(lookupUser);

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/ldap/ActiveDirectoryRealmTests.java

@@ -35,6 +35,7 @@ import org.elasticsearch.threadpool.ThreadPool;
 import org.elasticsearch.watcher.ResourceWatcherService;
 import org.elasticsearch.xpack.core.security.authc.AuthenticationResult;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.ldap.ActiveDirectorySessionFactorySettings;
 import org.elasticsearch.xpack.core.security.authc.ldap.LdapRealmSettings;
 import org.elasticsearch.xpack.core.security.authc.ldap.PoolingSessionFactorySettings;
@@ -166,7 +167,8 @@ public class ActiveDirectoryRealmTests extends ESTestCase {
      * the RealmConfig
      */
     private RealmConfig setupRealm(RealmConfig.RealmIdentifier realmIdentifier, Settings localSettings) {
-        final Settings mergedSettings = Settings.builder().put(globalSettings).put(localSettings).build();
+        final Settings mergedSettings = Settings.builder().put(globalSettings).put(localSettings)
+            .put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), "0").build();
         final Environment env = TestEnvironment.newEnvironment(mergedSettings);
         this.sslService = new SSLService(env);
         return new RealmConfig(

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/ldap/GroupsResolverTestCase.java

@@ -15,6 +15,7 @@ import org.elasticsearch.common.unit.TimeValue;
 import org.elasticsearch.common.util.concurrent.ThreadContext;
 import org.elasticsearch.env.TestEnvironment;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.security.authc.ldap.support.LdapSession.GroupsResolver;
 import org.elasticsearch.test.ESTestCase;
 import org.junit.After;
@@ -24,13 +25,17 @@ import java.nio.file.Path;
 import java.util.Collection;
 import java.util.List;
 
+import static org.elasticsearch.xpack.core.security.authc.RealmSettings.getFullSettingKey;
+
 public abstract class GroupsResolverTestCase extends ESTestCase {
 
     LDAPConnection ldapConnection;
 
     protected static RealmConfig config(RealmConfig.RealmIdentifier realmId, Settings settings) {
         if (settings.hasValue("path.home") == false) {
-            settings = Settings.builder().put(settings).put("path.home", createTempDir()).build();
+            settings = Settings.builder().put(settings).put("path.home", createTempDir())
+                .put(getFullSettingKey(realmId, RealmSettings.ORDER_SETTING), 0)
+                .build();
         }
         return new RealmConfig(realmId, settings, TestEnvironment.newEnvironment(settings), new ThreadContext(Settings.EMPTY));
     }

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/ldap/LdapRealmTests.java

@@ -116,6 +116,7 @@ public class LdapRealmTests extends LdapTestCase {
         Settings settings = Settings.builder()
             .put(defaultGlobalSettings)
             .put(buildLdapSettings(ldapUrls(), userTemplate, groupSearchBase, LdapSearchScope.SUB_TREE))
+            .put(getFullSettingKey(REALM_IDENTIFIER, RealmSettings.ORDER_SETTING), 0)
             .build();
         RealmConfig config = getRealmConfig(REALM_IDENTIFIER, settings);
         LdapSessionFactory ldapFactory = new LdapSessionFactory(config, sslService, threadPool);
@@ -147,6 +148,7 @@ public class LdapRealmTests extends LdapTestCase {
         Settings settings = Settings.builder()
                 .put(defaultGlobalSettings)
                 .put(buildLdapSettings(ldapUrls(), userTemplate, groupSearchBase, LdapSearchScope.ONE_LEVEL))
+                .put(getFullSettingKey(REALM_IDENTIFIER, RealmSettings.ORDER_SETTING), 0)
                 .build();
         RealmConfig config = getRealmConfig(REALM_IDENTIFIER, settings);
 
@@ -174,6 +176,7 @@ public class LdapRealmTests extends LdapTestCase {
         Settings settings = Settings.builder()
                 .put(defaultGlobalSettings)
                 .put(buildLdapSettings(ldapUrls(), userTemplate, groupSearchBase, LdapSearchScope.SUB_TREE))
+                .put(getFullSettingKey(REALM_IDENTIFIER, RealmSettings.ORDER_SETTING), 0)
                 .build();
         RealmConfig config = getRealmConfig(REALM_IDENTIFIER, settings);
 
@@ -201,6 +204,7 @@ public class LdapRealmTests extends LdapTestCase {
         Settings settings = Settings.builder()
                 .put(buildLdapSettings(ldapUrls(), userTemplate, groupSearchBase, LdapSearchScope.SUB_TREE))
                 .put(defaultGlobalSettings)
+                .put(getFullSettingKey(REALM_IDENTIFIER, RealmSettings.ORDER_SETTING), 0)
                 .build();
         RealmConfig config = getRealmConfig(REALM_IDENTIFIER, settings);
 
@@ -237,6 +241,7 @@ public class LdapRealmTests extends LdapTestCase {
                 .put(defaultGlobalSettings)
                 .put(buildLdapSettings(ldapUrls(), userTemplate, groupSearchBase, LdapSearchScope.SUB_TREE))
                 .put(getFullSettingKey(REALM_IDENTIFIER, CachingUsernamePasswordRealmSettings.CACHE_TTL_SETTING), -1)
+                .put(getFullSettingKey(REALM_IDENTIFIER, RealmSettings.ORDER_SETTING), 0)
                 .build();
         RealmConfig config = getRealmConfig(REALM_IDENTIFIER, settings);
 
@@ -268,6 +273,7 @@ public class LdapRealmTests extends LdapTestCase {
             // maybe disable caching
             builder.put(getFullSettingKey(REALM_IDENTIFIER, CachingUsernamePasswordRealmSettings.CACHE_TTL_SETTING), -1);
         }
+        builder.put(getFullSettingKey(REALM_IDENTIFIER, RealmSettings.ORDER_SETTING), 0);
 
         final Settings realmSettings = builder.build();
         final Environment env = TestEnvironment.newEnvironment(defaultGlobalSettings);
@@ -277,8 +283,12 @@ public class LdapRealmTests extends LdapTestCase {
         final DnRoleMapper roleMapper = buildGroupAsRoleMapper(resourceWatcherService);
         final LdapRealm ldap = new LdapRealm(config, ldapFactory, roleMapper, threadPool);
 
-        final MockLookupRealm mockLookup = new MockLookupRealm(new RealmConfig(new RealmConfig.RealmIdentifier("mock", "mock_lookup"),
-            defaultGlobalSettings, env, threadPool.getThreadContext()));
+        RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("mock", "mock_lookup");
+        final MockLookupRealm mockLookup = new MockLookupRealm(new RealmConfig(
+            realmIdentifier,
+            Settings.builder().put(defaultGlobalSettings)
+                .put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build(),
+            env, threadPool.getThreadContext()));
 
         ldap.initialize(Arrays.asList(ldap, mockLookup), licenseState);
         mockLookup.initialize(Arrays.asList(ldap, mockLookup), licenseState);
@@ -311,6 +321,7 @@ public class LdapRealmTests extends LdapTestCase {
                 .put(getFullSettingKey(identifier, SearchGroupsResolverSettings.BASE_DN), groupSearchBase)
                 .put(getFullSettingKey(identifier, SearchGroupsResolverSettings.SCOPE), LdapSearchScope.SUB_TREE)
                 .put(getFullSettingKey(identifier, SSLConfigurationSettings.VERIFICATION_MODE_SETTING_REALM), VerificationMode.CERTIFICATE)
+                .put(getFullSettingKey(identifier, RealmSettings.ORDER_SETTING), 0)
                 .build();
         RealmConfig config = getRealmConfig(identifier, settings);
         final SSLService ssl = new SSLService(config.env());
@@ -332,6 +343,7 @@ public class LdapRealmTests extends LdapTestCase {
                 .put(getFullSettingKey(identifier, SearchGroupsResolverSettings.BASE_DN), groupSearchBase)
                 .put(getFullSettingKey(identifier, SearchGroupsResolverSettings.SCOPE), LdapSearchScope.SUB_TREE)
                 .put(getFullSettingKey(identifier, SSLConfigurationSettings.VERIFICATION_MODE_SETTING_REALM), VerificationMode.CERTIFICATE)
+                .put(getFullSettingKey(identifier, RealmSettings.ORDER_SETTING), 0)
                 .build();
         final RealmConfig config = getRealmConfig(identifier, settings);
         SessionFactory sessionFactory = LdapRealm.sessionFactory(config, new SSLService(config.env()), threadPool);
@@ -353,6 +365,7 @@ public class LdapRealmTests extends LdapTestCase {
                 .put(getFullSettingKey(identifier, SearchGroupsResolverSettings.BASE_DN), "")
                 .put(getFullSettingKey(identifier, SearchGroupsResolverSettings.SCOPE), LdapSearchScope.SUB_TREE)
                 .put(getFullSettingKey(identifier, SSLConfigurationSettings.VERIFICATION_MODE_SETTING_REALM), VerificationMode.CERTIFICATE)
+                .put(getFullSettingKey(identifier, RealmSettings.ORDER_SETTING), 0)
                 .build();
         RealmConfig config = getRealmConfig(identifier, settings);
         IllegalArgumentException e = expectThrows(IllegalArgumentException.class,
@@ -372,6 +385,7 @@ public class LdapRealmTests extends LdapTestCase {
                 .put(getFullSettingKey(identifier, SearchGroupsResolverSettings.BASE_DN), "")
                 .put(getFullSettingKey(identifier, SearchGroupsResolverSettings.SCOPE), LdapSearchScope.SUB_TREE)
                 .put(getFullSettingKey(identifier, SSLConfigurationSettings.VERIFICATION_MODE_SETTING_REALM), VerificationMode.CERTIFICATE)
+                .put(getFullSettingKey(identifier, RealmSettings.ORDER_SETTING), 0)
                 .build();
         RealmConfig config = getRealmConfig(identifier, settings);
         IllegalArgumentException e = expectThrows(IllegalArgumentException.class,
@@ -491,6 +505,7 @@ public class LdapRealmTests extends LdapTestCase {
         Settings settings = Settings.builder()
             .put(defaultGlobalSettings)
             .put(buildLdapSettings(new String[]{url.toString()}, userTemplate, groupSearchBase, LdapSearchScope.SUB_TREE))
+            .put(getFullSettingKey(REALM_IDENTIFIER, RealmSettings.ORDER_SETTING), 0)
             .build();
         RealmConfig config = getRealmConfig(REALM_IDENTIFIER, settings);
         LdapSessionFactory ldapFactory = new LdapSessionFactory(config, sslService, threadPool);

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/ldap/SearchGroupsResolverInMemoryTests.java

@@ -19,6 +19,7 @@ import org.elasticsearch.common.unit.TimeValue;
 import org.elasticsearch.common.util.concurrent.ThreadContext;
 import org.elasticsearch.env.TestEnvironment;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.ldap.PoolingSessionFactorySettings;
 import org.elasticsearch.xpack.core.security.authc.ldap.SearchGroupsResolverSettings;
 import org.elasticsearch.xpack.core.security.authc.ldap.support.LdapSearchScope;
@@ -167,7 +168,9 @@ public class SearchGroupsResolverInMemoryTests extends LdapTestCase {
         if (settings.hasValue("path.home") == false) {
             settings = Settings.builder().put(settings).put("path.home", createTempDir()).build();
         }
-        return new RealmConfig(REALM_IDENTIFIER, settings, TestEnvironment.newEnvironment(settings), new ThreadContext(settings));
+        return new RealmConfig(REALM_IDENTIFIER,
+            Settings.builder().put(settings).put(getFullSettingKey(REALM_IDENTIFIER, RealmSettings.ORDER_SETTING), 0).build(),
+            TestEnvironment.newEnvironment(settings), new ThreadContext(settings));
     }
 
 }

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/ldap/support/LdapLoadBalancingTests.java

@@ -14,6 +14,7 @@ import org.elasticsearch.common.util.concurrent.ThreadContext;
 import org.elasticsearch.env.TestEnvironment;
 import org.elasticsearch.test.ESTestCase;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.ldap.support.LdapLoadBalancingSettings;
 
 import static org.elasticsearch.xpack.core.security.authc.RealmSettings.getFullSettingKey;
@@ -40,6 +41,7 @@ public class LdapLoadBalancingTests extends ESTestCase {
         return Settings.builder()
                 .put(getFullSettingKey(REALM_IDENTIFIER, LdapLoadBalancingSettings.LOAD_BALANCE_TYPE_SETTING), loadBalancerType)
                 .put("path.home", createTempDir())
+                .put(getFullSettingKey(REALM_IDENTIFIER, RealmSettings.ORDER_SETTING), 0)
                 .build();
     }
 

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/ldap/support/LdapMetaDataResolverTests.java

@@ -42,6 +42,7 @@ public class LdapMetaDataResolverTests extends ESTestCase {
                 .putList(RealmSettings.getFullSettingKey(realmId.getName(),
                         LdapMetaDataResolverSettings.ADDITIONAL_META_DATA_SETTING.apply(LdapRealmSettings.LDAP_TYPE)),
                         "cn", "uid")
+                .put(RealmSettings.getFullSettingKey(realmId, RealmSettings.ORDER_SETTING), 0)
                 .build();
         RealmConfig config = new RealmConfig(realmId,
                 settings, TestEnvironment.newEnvironment(settings), new ThreadContext(settings));

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/ldap/support/LdapTestCase.java

@@ -28,6 +28,7 @@ import org.elasticsearch.test.ESTestCase;
 import org.elasticsearch.watcher.ResourceWatcherService;
 import org.elasticsearch.xpack.core.XPackSettings;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.ldap.LdapSessionFactorySettings;
 import org.elasticsearch.xpack.core.security.authc.ldap.SearchGroupsResolverSettings;
 import org.elasticsearch.xpack.core.security.authc.ldap.support.LdapLoadBalancingSettings;
@@ -172,6 +173,7 @@ public abstract class LdapTestCase extends ESTestCase {
         if (serverSetType != null) {
             builder.put(getFullSettingKey(realmId, LdapLoadBalancingSettings.LOAD_BALANCE_TYPE_SETTING), serverSetType.toString());
         }
+        builder.put(getFullSettingKey(realmId, RealmSettings.ORDER_SETTING), 0);
         return builder.build();
     }
 
@@ -192,6 +194,7 @@ public abstract class LdapTestCase extends ESTestCase {
         Settings settings = Settings.builder()
                 .put(getFullSettingKey(REALM_IDENTIFIER, DnRoleMapperSettings.USE_UNMAPPED_GROUPS_AS_ROLES_SETTING), true)
                 .put("path.home", createTempDir())
+                .put(getFullSettingKey(REALM_IDENTIFIER, RealmSettings.ORDER_SETTING), 0)
                 .build();
         RealmConfig config = new RealmConfig(REALM_IDENTIFIER, settings,
                 TestEnvironment.newEnvironment(settings), new ThreadContext(Settings.EMPTY));

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/ldap/support/SessionFactoryTests.java

@@ -19,6 +19,7 @@ import org.elasticsearch.test.ESTestCase;
 import org.elasticsearch.threadpool.TestThreadPool;
 import org.elasticsearch.threadpool.ThreadPool;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.ldap.support.SessionFactorySettings;
 import org.elasticsearch.xpack.core.ssl.SSLConfigurationSettings;
 import org.elasticsearch.xpack.core.ssl.SSLService;
@@ -49,8 +50,12 @@ public class SessionFactoryTests extends ESTestCase {
     }
 
     public void testConnectionFactoryReturnsCorrectLDAPConnectionOptionsWithDefaultSettings() throws Exception {
-        final Environment environment = TestEnvironment.newEnvironment(Settings.builder().put("path.home", createTempDir()).build());
-        RealmConfig realmConfig = new RealmConfig(new RealmConfig.RealmIdentifier("ldap", "conn_settings"),
+        final RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("ldap", "conn_settings");
+        final Environment environment = TestEnvironment.newEnvironment(
+            Settings.builder().put("path.home", createTempDir())
+                .put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build());
+        RealmConfig realmConfig = new RealmConfig(
+            realmIdentifier,
                 environment.settings(), environment, new ThreadContext(Settings.EMPTY));
         LDAPConnectionOptions options = SessionFactory.connectionOptions(realmConfig, new SSLService(environment),
                 logger);
@@ -69,6 +74,7 @@ public class SessionFactoryTests extends ESTestCase {
                 .put(getFullSettingKey(realmId, SessionFactorySettings.HOSTNAME_VERIFICATION_SETTING), "false")
                 .put(getFullSettingKey(realmId, SessionFactorySettings.TIMEOUT_TCP_READ_SETTING), "20ms")
                 .put(getFullSettingKey(realmId, SessionFactorySettings.FOLLOW_REFERRALS_SETTING), "false")
+                .put(getFullSettingKey(realmId, RealmSettings.ORDER_SETTING), 0)
                 .put("path.home", pathHome)
                 .build();
 
@@ -86,6 +92,7 @@ public class SessionFactoryTests extends ESTestCase {
         settings = Settings.builder()
                 .put(getFullSettingKey(realmId, SSLConfigurationSettings.VERIFICATION_MODE_SETTING_REALM), VerificationMode.CERTIFICATE)
                 .put("path.home", pathHome)
+                .put(getFullSettingKey(realmId, RealmSettings.ORDER_SETTING), 0)
                 .build();
         realmConfig = new RealmConfig(realmId, settings, environment, new ThreadContext(settings));
         options = SessionFactory.connectionOptions(realmConfig, new SSLService(TestEnvironment.newEnvironment(settings)), logger);
@@ -96,6 +103,7 @@ public class SessionFactoryTests extends ESTestCase {
             settings = Settings.builder()
                     .put(getFullSettingKey(realmId, SSLConfigurationSettings.VERIFICATION_MODE_SETTING_REALM), VerificationMode.NONE)
                     .put("path.home", pathHome)
+                    .put(getFullSettingKey(realmId, RealmSettings.ORDER_SETTING), 0)
                     .build();
             environment = TestEnvironment.newEnvironment(settings);
             realmConfig = new RealmConfig(realmId, settings, environment, new ThreadContext(settings));
@@ -106,6 +114,7 @@ public class SessionFactoryTests extends ESTestCase {
         settings = Settings.builder()
                 .put(getFullSettingKey(realmId, SSLConfigurationSettings.VERIFICATION_MODE_SETTING_REALM), VerificationMode.FULL)
                 .put("path.home", pathHome)
+                .put(getFullSettingKey(realmId, RealmSettings.ORDER_SETTING), 0)
                 .build();
         environment = TestEnvironment.newEnvironment(settings);
         realmConfig = new RealmConfig(realmId, settings, environment, new ThreadContext(settings));
@@ -130,6 +139,7 @@ public class SessionFactoryTests extends ESTestCase {
                 Settings.builder()
                         .put(getFullSettingKey(realmIdentifier, SessionFactorySettings.URLS_SETTING), "ldap://localhost:389")
                         .put(global)
+                        .put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0)
                         .build(),
                 TestEnvironment.newEnvironment(global), new ThreadContext(Settings.EMPTY));
         return new SessionFactory(realmConfig, null, threadPool) {

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/oidc/OpenIdConnectRealmSettingsTests.java

@@ -14,6 +14,7 @@ import org.elasticsearch.env.Environment;
 import org.elasticsearch.env.TestEnvironment;
 import org.elasticsearch.test.ESTestCase;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.oidc.OpenIdConnectRealmSettings;
 import org.hamcrest.Matchers;
 import org.junit.Before;
@@ -271,10 +272,13 @@ public class OpenIdConnectRealmSettingsTests extends ESTestCase {
     }
 
     private RealmConfig buildConfig(Settings realmSettings) {
+        RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("oidc", REALM_NAME);
         final Settings settings = Settings.builder()
             .put("path.home", createTempDir())
-            .put(realmSettings).build();
+            .put(realmSettings)
+            .put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0)
+            .build();
         final Environment env = TestEnvironment.newEnvironment(settings);
-        return new RealmConfig(new RealmConfig.RealmIdentifier("oidc", REALM_NAME), settings, env, threadContext);
+        return new RealmConfig(realmIdentifier, settings, env, threadContext);
     }
 }

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/oidc/OpenIdConnectRealmTests.java

@@ -21,6 +21,7 @@ import org.elasticsearch.xpack.core.security.action.oidc.OpenIdConnectPrepareAut
 import org.elasticsearch.xpack.core.security.authc.AuthenticationResult;
 import org.elasticsearch.xpack.core.security.authc.Realm;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.oidc.OpenIdConnectRealmSettings;
 import org.elasticsearch.xpack.core.security.authc.support.DelegatedAuthorizationSettings;
 import org.elasticsearch.xpack.core.security.user.User;
@@ -308,8 +309,12 @@ public class OpenIdConnectRealmTests extends OpenIdConnectTestCase {
                                                       boolean useAuthorizingRealm
         ,String authenticatingRealm)
         throws Exception {
+        RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("mock", "mock_lookup");
         final MockLookupRealm lookupRealm = new MockLookupRealm(
-            new RealmConfig(new RealmConfig.RealmIdentifier("mock", "mock_lookup"), globalSettings, env, threadContext));
+            new RealmConfig(realmIdentifier,
+                Settings.builder().put(globalSettings)
+                    .put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build(),
+                env, threadContext));
         final OpenIdConnectAuthenticator authenticator = mock(OpenIdConnectAuthenticator.class);
 
         final Settings.Builder builder = getBasicRealmSettings();

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/oidc/OpenIdConnectTestCase.java

@@ -19,6 +19,7 @@ import org.elasticsearch.env.Environment;
 import org.elasticsearch.env.TestEnvironment;
 import org.elasticsearch.test.ESTestCase;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.oidc.OpenIdConnectRealmSettings;
 
 import java.io.IOException;
@@ -37,6 +38,7 @@ public abstract class OpenIdConnectTestCase extends ESTestCase {
     protected static final String REALM_NAME = "oidc-realm";
 
     protected static Settings.Builder getBasicRealmSettings() {
+        final RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier(OpenIdConnectRealmSettings.TYPE, REALM_NAME);
         return Settings.builder()
             .put(getFullSettingKey(REALM_NAME, OpenIdConnectRealmSettings.OP_AUTHORIZATION_ENDPOINT), "https://op.example.org/login")
             .put(getFullSettingKey(REALM_NAME, OpenIdConnectRealmSettings.OP_TOKEN_ENDPOINT), "https://op.example.org/token")
@@ -52,6 +54,7 @@ public abstract class OpenIdConnectTestCase extends ESTestCase {
             .put(getFullSettingKey(REALM_NAME, OpenIdConnectRealmSettings.GROUPS_CLAIM.getClaim()), "groups")
             .put(getFullSettingKey(REALM_NAME, OpenIdConnectRealmSettings.MAIL_CLAIM.getClaim()), "mail")
             .put(getFullSettingKey(REALM_NAME, OpenIdConnectRealmSettings.NAME_CLAIM.getClaim()), "name")
+            .put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0)
             .setSecureSettings(getSecureSettings());
     }
 
@@ -87,11 +90,14 @@ public abstract class OpenIdConnectTestCase extends ESTestCase {
     }
 
     protected RealmConfig buildConfig(Settings realmSettings, ThreadContext threadContext) {
+        RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("oidc", REALM_NAME);
         final Settings settings = Settings.builder()
             .put("path.home", createTempDir())
-            .put(realmSettings).build();
+            .put(realmSettings)
+            .put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0)
+            .build();
         final Environment env = TestEnvironment.newEnvironment(settings);
-        return new RealmConfig(new RealmConfig.RealmIdentifier("oidc", REALM_NAME), settings, env, threadContext);
+        return new RealmConfig(realmIdentifier, settings, env, threadContext);
     }
 
     public static void writeJwkSetToFile(Path file) throws IOException {

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/pki/PkiAuthDelegationIntegTests.java

@@ -54,19 +54,19 @@ public class PkiAuthDelegationIntegTests extends SecurityIntegTestCase {
                 .put(super.nodeSettings(nodeOrdinal))
                 .put(XPackSettings.TOKEN_SERVICE_ENABLED_SETTING.getKey(), true)
                 // pki1 does not allow delegation
-                .put("xpack.security.authc.realms.pki.pki1.order", "1")
+                .put("xpack.security.authc.realms.pki.pki1.order", "2")
                 .putList("xpack.security.authc.realms.pki.pki1.certificate_authorities",
                     getDataPath("/org/elasticsearch/xpack/security/action/pki_delegation/testRootCA.crt").toString())
                 .put("xpack.security.authc.realms.pki.pki1.files.role_mapping", getDataPath("role_mapping.yml"))
                 // pki2 allows delegation but has a non-matching username pattern
-                .put("xpack.security.authc.realms.pki.pki2.order", "2")
+                .put("xpack.security.authc.realms.pki.pki2.order", "3")
                 .putList("xpack.security.authc.realms.pki.pki2.certificate_authorities",
                     getDataPath("/org/elasticsearch/xpack/security/action/pki_delegation/testRootCA.crt").toString())
                 .put("xpack.security.authc.realms.pki.pki2.username_pattern", "CN=MISMATCH(.*?)(?:,|$)")
                 .put("xpack.security.authc.realms.pki.pki2.delegation.enabled", true)
                 .put("xpack.security.authc.realms.pki.pki2.files.role_mapping", getDataPath("role_mapping.yml"))
                 // pki3 allows delegation and the username pattern (default) matches
-                .put("xpack.security.authc.realms.pki.pki3.order", "3")
+                .put("xpack.security.authc.realms.pki.pki3.order", "4")
                 .putList("xpack.security.authc.realms.pki.pki3.certificate_authorities",
                     getDataPath("/org/elasticsearch/xpack/security/action/pki_delegation/testRootCA.crt").toString())
                 .put("xpack.security.authc.realms.pki.pki3.delegation.enabled", true)

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/pki/PkiAuthenticationTests.java

@@ -55,14 +55,14 @@ public class PkiAuthenticationTests extends SecuritySingleNodeTestCase {
         builder.put("xpack.security.http.ssl.enabled", true)
             .put("xpack.security.http.ssl.client_authentication", sslClientAuth)
             .put("xpack.security.authc.realms.file.file.order", "0")
-            .put("xpack.security.authc.realms.pki.pki1.order", "1")
+            .put("xpack.security.authc.realms.pki.pki1.order", "2")
             .putList("xpack.security.authc.realms.pki.pki1.certificate_authorities",
                 getDataPath("/org/elasticsearch/xpack/security/transport/ssl/certs/simple/testclient.crt").toString())
             .put("xpack.security.authc.realms.pki.pki1.files.role_mapping", getDataPath("role_mapping.yml"))
             .put("xpack.security.authc.realms.pki.pki1.files.role_mapping", getDataPath("role_mapping.yml"))
             // pki1 never authenticates because of the principal pattern
             .put("xpack.security.authc.realms.pki.pki1.username_pattern", "CN=(MISMATCH.*?)(?:,|$)")
-            .put("xpack.security.authc.realms.pki.pki2.order", "2")
+            .put("xpack.security.authc.realms.pki.pki2.order", "3")
             .putList("xpack.security.authc.realms.pki.pki2.certificate_authorities",
                 getDataPath("/org/elasticsearch/xpack/security/transport/ssl/certs/simple/testnode.crt").toString(),
                 getDataPath("/org/elasticsearch/xpack/security/transport/ssl/certs/simple/testnode_ec.crt").toString())

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/pki/PkiOptionalClientAuthTests.java

@@ -57,7 +57,7 @@ public class PkiOptionalClientAuthTests extends SecuritySingleNodeTestCase {
             .put("xpack.security.http.ssl.certificate",
                 getDataPath("/org/elasticsearch/xpack/security/transport/ssl/certs/simple/testnode.crt"))
             .put("xpack.security.authc.realms.file.file.order", "0")
-            .put("xpack.security.authc.realms.pki.pki1.order", "1")
+            .put("xpack.security.authc.realms.pki.pki1.order", "2")
             .put("xpack.security.authc.realms.pki.pki1.truststore.path",
                 getDataPath("/org/elasticsearch/xpack/security/transport/ssl/certs/simple/truststore-testnode-only.jks"))
             .put("xpack.security.authc.realms.pki.pki1.files.role_mapping", getDataPath("role_mapping.yml"))

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/pki/PkiRealmTests.java

@@ -23,6 +23,7 @@ import org.elasticsearch.xpack.core.security.authc.AuthenticationResult;
 import org.elasticsearch.xpack.core.security.authc.InternalRealmsSettings;
 import org.elasticsearch.xpack.core.security.authc.Realm;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.pki.PkiRealmSettings;
 import org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken;
 import org.elasticsearch.xpack.core.security.support.NoOpLogger;
@@ -70,16 +71,19 @@ public class PkiRealmTests extends ESTestCase {
 
     @Before
     public void setup() throws Exception {
+        RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier(PkiRealmSettings.TYPE, REALM_NAME);
         globalSettings = Settings.builder()
                 .put("path.home", createTempDir())
+                .put(RealmSettings.getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0)
                 .build();
         licenseState = mock(XPackLicenseState.class);
         when(licenseState.isAuthorizationRealmAllowed()).thenReturn(true);
     }
 
     public void testTokenSupport() throws Exception {
-        RealmConfig config = new RealmConfig(new RealmConfig.RealmIdentifier("pki", "my_pki"), globalSettings,
-                TestEnvironment.newEnvironment(globalSettings), new ThreadContext(globalSettings));
+        RealmConfig config = new RealmConfig(new RealmConfig.RealmIdentifier(PkiRealmSettings.TYPE, REALM_NAME),
+            globalSettings,
+            TestEnvironment.newEnvironment(globalSettings), new ThreadContext(globalSettings));
         PkiRealm realm = new PkiRealm(config, mock(UserRoleMapper.class));
 
         assertRealmUsageStats(realm, false, false, true, false);
@@ -95,7 +99,7 @@ public class PkiRealmTests extends ESTestCase {
         X509Certificate certificate = readCert(getDataPath("/org/elasticsearch/xpack/security/transport/ssl/certs/simple/testnode.crt"));
         ThreadContext threadContext = new ThreadContext(Settings.EMPTY);
         threadContext.putTransient(PkiRealm.PKI_CERT_HEADER_NAME, new X509Certificate[]{certificate});
-        PkiRealm realm = new PkiRealm(new RealmConfig(new RealmConfig.RealmIdentifier("pki", "my_pki"), globalSettings,
+        PkiRealm realm = new PkiRealm(new RealmConfig(new RealmConfig.RealmIdentifier(PkiRealmSettings.TYPE, REALM_NAME), globalSettings,
                 TestEnvironment.newEnvironment(globalSettings), threadContext), mock(UserRoleMapper.class));
 
         X509AuthenticationToken token = realm.token(threadContext);
@@ -178,7 +182,7 @@ public class PkiRealmTests extends ESTestCase {
     }
 
     private PkiRealm buildRealm(UserRoleMapper roleMapper, Settings settings, Realm... otherRealms) {
-        final RealmConfig config = new RealmConfig(new RealmConfig.RealmIdentifier("pki", REALM_NAME), settings,
+        final RealmConfig config = new RealmConfig(new RealmConfig.RealmIdentifier(PkiRealmSettings.TYPE, REALM_NAME), settings,
             TestEnvironment.newEnvironment(settings), new ThreadContext(settings));
         PkiRealm realm = new PkiRealm(config, roleMapper);
         List<Realm> allRealms = CollectionUtils.arrayAsArrayList(otherRealms);
@@ -269,7 +273,7 @@ public class PkiRealmTests extends ESTestCase {
                 .put("xpack.security.authc.realms.pki.my_pki.delegation.enabled", true)
                 .build();
         IllegalStateException e = expectThrows(IllegalStateException.class,
-                () -> new PkiRealm(new RealmConfig(new RealmConfig.RealmIdentifier("pki", "my_pki"), settings,
+                () -> new PkiRealm(new RealmConfig(new RealmConfig.RealmIdentifier(PkiRealmSettings.TYPE, REALM_NAME), settings,
                         TestEnvironment.newEnvironment(globalSettings), threadContext), mock(UserRoleMapper.class)));
         assertThat(e.getMessage(),
                 is("PKI realms with delegation enabled require a trust configuration "
@@ -286,7 +290,7 @@ public class PkiRealmTests extends ESTestCase {
                 .put("xpack.security.authc.token.enabled", true)
                 .build();
         IllegalStateException e = expectThrows(IllegalStateException.class,
-                () -> new PkiRealm(new RealmConfig(new RealmConfig.RealmIdentifier("pki", "my_pki"), settings,
+                () -> new PkiRealm(new RealmConfig(new RealmConfig.RealmIdentifier(PkiRealmSettings.TYPE, REALM_NAME), settings,
                         TestEnvironment.newEnvironment(globalSettings), threadContext), mock(UserRoleMapper.class)));
         assertThat(e.getMessage(),
                 is("PKI realms with delegation enabled require a trust configuration "
@@ -384,29 +388,30 @@ public class PkiRealmTests extends ESTestCase {
         assumeFalse("Can't run in a FIPS JVM, JKS keystores can't be used", inFipsJvm());
         Settings settings = Settings.builder()
                 .put(globalSettings)
-                .put("xpack.security.authc.realms.pki.mypki.truststore.path",
+                .put("xpack.security.authc.realms.pki.my_pki.truststore.path",
                         getDataPath("/org/elasticsearch/xpack/security/transport/ssl/certs/simple/testnode-client-profile.jks"))
                 .build();
         IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () ->
-                new PkiRealm(new RealmConfig(new RealmConfig.RealmIdentifier("pki", "mypki"), settings,
-                        TestEnvironment.newEnvironment(settings), new ThreadContext(settings)), mock(UserRoleMapper.class))
+                new PkiRealm(new RealmConfig(new RealmConfig.RealmIdentifier(PkiRealmSettings.TYPE, REALM_NAME), settings,
+                        TestEnvironment.newEnvironment(settings), new ThreadContext(settings)),
+                    mock(UserRoleMapper.class))
         );
-        assertThat(e.getMessage(), containsString("Neither [xpack.security.authc.realms.pki.mypki.truststore.secure_password] or [" +
-                    "xpack.security.authc.realms.pki.mypki.truststore.password] is configured"));
+        assertThat(e.getMessage(), containsString("Neither [xpack.security.authc.realms.pki.my_pki.truststore.secure_password] or [" +
+                    "xpack.security.authc.realms.pki.my_pki.truststore.password] is configured"));
     }
 
     public void testTruststorePathWithLegacyPasswordDoesNotThrow() throws Exception {
         assumeFalse("Can't run in a FIPS JVM, JKS keystores can't be used", inFipsJvm());
         Settings settings = Settings.builder()
                 .put(globalSettings)
-                .put("xpack.security.authc.realms.pki.mypki.truststore.path",
+                .put("xpack.security.authc.realms.pki.my_pki.truststore.path",
                         getDataPath("/org/elasticsearch/xpack/security/transport/ssl/certs/simple/testnode-client-profile.jks"))
-                .put("xpack.security.authc.realms.pki.mypki.truststore.password", "testnode-client-profile")
+                .put("xpack.security.authc.realms.pki.my_pki.truststore.password", "testnode-client-profile")
                 .build();
-        new PkiRealm(new RealmConfig(new RealmConfig.RealmIdentifier("pki", "mypki"), settings,
+        new PkiRealm(new RealmConfig(new RealmConfig.RealmIdentifier(PkiRealmSettings.TYPE, REALM_NAME), settings,
                 TestEnvironment.newEnvironment(settings), new ThreadContext(settings)), mock(UserRoleMapper.class));
         assertSettingDeprecationsAndWarnings(new Setting[]{
-                PkiRealmSettings.LEGACY_TRUST_STORE_PASSWORD.getConcreteSettingForNamespace("mypki")
+                PkiRealmSettings.LEGACY_TRUST_STORE_PASSWORD.getConcreteSettingForNamespace(REALM_NAME)
         });
     }
 
@@ -473,8 +478,12 @@ public class PkiRealmTests extends ESTestCase {
         String parsedPrincipal = PkiRealm.getPrincipalFromSubjectDN(Pattern.compile(PkiRealmSettings.DEFAULT_USERNAME_PATTERN), token,
                 NoOpLogger.INSTANCE);
 
-        final MockLookupRealm otherRealm = new MockLookupRealm(new RealmConfig(new RealmConfig.RealmIdentifier("mock", "other_realm"),
-            globalSettings, TestEnvironment.newEnvironment(globalSettings), new ThreadContext(globalSettings)));
+        RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("mock", "other_realm");
+        final MockLookupRealm otherRealm = new MockLookupRealm(new RealmConfig(
+            realmIdentifier,
+            Settings.builder().put(globalSettings)
+                .put(RealmSettings.getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build(),
+            TestEnvironment.newEnvironment(globalSettings), new ThreadContext(globalSettings)));
         final User lookupUser = new User(parsedPrincipal);
         otherRealm.registerUser(lookupUser);
 

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/saml/SamlMetadataCommandTests.java

@@ -189,9 +189,11 @@ public class SamlMetadataCommandTests extends SamlTestCase {
         final KeyStoreWrapper usedKeyStore = randomFrom(keyStore, passwordProtectedKeystore);
         final Settings settings = Settings.builder()
                 .put("path.home", createTempDir())
+                .put(RealmSettings.PREFIX + "saml.saml_a.order", 1)
                 .put(RealmSettings.PREFIX + "saml.saml_a.type", "saml")
                 .put(RealmSettings.PREFIX + "saml.saml_a.sp.entity_id", "https://saml.a/")
                 .put(RealmSettings.PREFIX + "saml.saml_a.sp.acs", "https://saml.a/acs")
+                .put(RealmSettings.PREFIX + "saml.saml_b.order", 2)
                 .put(RealmSettings.PREFIX + "saml.saml_b.type", "saml")
                 .put(RealmSettings.PREFIX + "saml.saml_b.sp.entity_id", "https://saml.b/")
                 .put(RealmSettings.PREFIX + "saml.saml_b.sp.acs", "https://saml.b/acs")
@@ -221,6 +223,7 @@ public class SamlMetadataCommandTests extends SamlTestCase {
         final KeyStoreWrapper usedKeyStore = randomFrom(keyStore, passwordProtectedKeystore);
         final Settings settings = Settings.builder()
                 .put("path.home", createTempDir())
+                .put(RealmSettings.PREFIX + "saml.saml1.order", 1)
                 .put(RealmSettings.PREFIX + "saml.saml1.type", "saml")
                 .put(RealmSettings.PREFIX + "saml.saml1.sp.entity_id", "https://saml.example.com/")
                 .put(RealmSettings.PREFIX + "saml.saml1.sp.acs", "https://saml.example.com/")
@@ -275,6 +278,7 @@ public class SamlMetadataCommandTests extends SamlTestCase {
         final KeyStoreWrapper usedKeyStore = randomFrom(keyStore, passwordProtectedKeystore);
         final Settings settings = Settings.builder()
                 .put("path.home", createTempDir())
+                .put(RealmSettings.PREFIX + "saml.saml1.order", 1)
                 .put(RealmSettings.PREFIX + "saml.saml1.type", "saml")
                 .put(RealmSettings.PREFIX + "saml.saml1.sp.entity_id", "https://saml.example.com/")
                 .put(RealmSettings.PREFIX + "saml.saml1.sp.acs", "https://saml.example.com/")

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/saml/SamlRealmTests.java

@@ -246,8 +246,11 @@ public class SamlRealmTests extends SamlTestCase {
         final String nameIdValue = principalIsEmailAddress ? "clint.barton@shield.gov" : "clint.barton";
         final String uidValue = principalIsEmailAddress ? "cbarton@shield.gov" : "cbarton";
 
+        final RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("mock", "mock_lookup");
         final MockLookupRealm lookupRealm = new MockLookupRealm(
-            new RealmConfig(new RealmConfig.RealmIdentifier("mock","mock_lookup"), globalSettings, env, threadContext));
+            new RealmConfig(realmIdentifier,
+                Settings.builder().put(globalSettings).put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build(),
+                env, threadContext));
 
         final Settings.Builder settingsBuilder = Settings.builder()
                 .put(getFullSettingKey(REALM_NAME, SamlRealmSettings.PRINCIPAL_ATTRIBUTE.getAttribute()), useNameId ? "nameid" : "uid")
@@ -718,12 +721,19 @@ public class SamlRealmTests extends SamlTestCase {
                 .put("path.home", createTempDir())
                 .put(realmSettings).build();
         final Environment env = TestEnvironment.newEnvironment(settings);
-        return new RealmConfig(new RealmConfig.RealmIdentifier("saml", REALM_NAME), settings, env, threadContext);
+        final RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("saml", REALM_NAME);
+        return new RealmConfig(realmIdentifier,
+            Settings.builder().put(settings).put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build(),
+            env, threadContext);
     }
 
     private RealmConfig realmConfigFromGlobalSettings(Settings globalSettings) {
         final Environment env = TestEnvironment.newEnvironment(globalSettings);
-        return new RealmConfig(new RealmConfig.RealmIdentifier("saml", REALM_NAME), globalSettings, env, new ThreadContext(globalSettings));
+        final RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("saml", REALM_NAME);
+        return new RealmConfig(realmIdentifier,
+            Settings.builder().put(globalSettings).put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build(),
+            env,
+            new ThreadContext(globalSettings));
     }
 
     private void assertIdp1MetadataParsedCorrectly(EntityDescriptor descriptor) {

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/support/CachingUsernamePasswordRealmTests.java

@@ -35,6 +35,7 @@ import java.util.concurrent.atomic.AtomicInteger;
 import java.util.concurrent.atomic.AtomicReference;
 
 import static java.util.Collections.emptyMap;
+import static org.elasticsearch.xpack.core.security.authc.RealmSettings.getFullSettingKey;
 import static org.hamcrest.Matchers.arrayContaining;
 import static org.hamcrest.Matchers.containsString;
 import static org.hamcrest.Matchers.equalTo;
@@ -70,9 +71,10 @@ public class CachingUsernamePasswordRealmTests extends ESTestCase {
         final RealmConfig.RealmIdentifier identifier = new RealmConfig.RealmIdentifier("caching", "test_realm");
         Settings settings = Settings.builder()
             .put(globalSettings)
-            .put(RealmSettings.getFullSettingKey(identifier, CachingUsernamePasswordRealmSettings.CACHE_HASH_ALGO_SETTING), cachingHashAlgo)
-            .put(RealmSettings.getFullSettingKey(identifier, CachingUsernamePasswordRealmSettings.CACHE_MAX_USERS_SETTING), maxUsers)
-            .put(RealmSettings.getFullSettingKey(identifier, CachingUsernamePasswordRealmSettings.CACHE_TTL_SETTING), ttl)
+            .put(getFullSettingKey(identifier, CachingUsernamePasswordRealmSettings.CACHE_HASH_ALGO_SETTING), cachingHashAlgo)
+            .put(getFullSettingKey(identifier, CachingUsernamePasswordRealmSettings.CACHE_MAX_USERS_SETTING), maxUsers)
+            .put(getFullSettingKey(identifier, CachingUsernamePasswordRealmSettings.CACHE_TTL_SETTING), ttl)
+            .put(getFullSettingKey(identifier, RealmSettings.ORDER_SETTING), 0)
             .build();
 
         RealmConfig config = new RealmConfig(identifier, settings,
@@ -95,7 +97,8 @@ public class CachingUsernamePasswordRealmTests extends ESTestCase {
         final RealmConfig.RealmIdentifier identifier = new RealmConfig.RealmIdentifier("caching", "test_realm");
         final Settings settings = Settings.builder()
                 .put(globalSettings)
-                .put(RealmSettings.getFullSettingKey(identifier, CachingUsernamePasswordRealmSettings.CACHE_TTL_SETTING), -1)
+                .put(getFullSettingKey(identifier, CachingUsernamePasswordRealmSettings.CACHE_TTL_SETTING), -1)
+                .put(getFullSettingKey(identifier, RealmSettings.ORDER_SETTING), 0)
                 .build();
 
         final RealmConfig config =
@@ -275,7 +278,8 @@ public class CachingUsernamePasswordRealmTests extends ESTestCase {
         final RealmConfig.RealmIdentifier identifier = new RealmConfig.RealmIdentifier("caching", "test_cache_ttl");
         Settings settings = Settings.builder()
                 .put(globalSettings)
-                .put(RealmSettings.getFullSettingKey(identifier, CachingUsernamePasswordRealmSettings.CACHE_TTL_SETTING), ttl)
+                .put(getFullSettingKey(identifier, CachingUsernamePasswordRealmSettings.CACHE_TTL_SETTING), ttl)
+                .put(getFullSettingKey(identifier, RealmSettings.ORDER_SETTING), 0)
                 .build();
         RealmConfig config = new RealmConfig(identifier, settings,
                 TestEnvironment.newEnvironment(globalSettings), new ThreadContext(Settings.EMPTY));
@@ -306,7 +310,8 @@ public class CachingUsernamePasswordRealmTests extends ESTestCase {
         final RealmConfig.RealmIdentifier identifier = new RealmConfig.RealmIdentifier("caching", "test_cache_ttl");
         Settings settings = Settings.builder()
                 .put(globalSettings)
-                .put(RealmSettings.getFullSettingKey(identifier, CachingUsernamePasswordRealmSettings.CACHE_TTL_SETTING), ttl)
+                .put(getFullSettingKey(identifier, CachingUsernamePasswordRealmSettings.CACHE_TTL_SETTING), ttl)
+                .put(getFullSettingKey(identifier, RealmSettings.ORDER_SETTING), 0)
                 .build();
         RealmConfig config = new RealmConfig(identifier, settings,
                 TestEnvironment.newEnvironment(globalSettings), new ThreadContext(Settings.EMPTY));
@@ -412,7 +417,11 @@ public class CachingUsernamePasswordRealmTests extends ESTestCase {
         final AtomicInteger authCounter = new AtomicInteger(0);
         final Hasher pwdHasher = Hasher.resolve(randomFrom("pbkdf2", "pbkdf2_1000", "bcrypt", "bcrypt9"));
         final String passwordHash = new String(pwdHasher.hash(password));
-        RealmConfig config = new RealmConfig(new RealmConfig.RealmIdentifier("caching", "test_realm"), globalSettings,
+        final RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("caching", "test_realm");
+        RealmConfig config = new RealmConfig(
+            realmIdentifier,
+            Settings.builder().put(globalSettings)
+                .put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build(),
             TestEnvironment.newEnvironment(globalSettings), new ThreadContext(Settings.EMPTY));
         final CachingUsernamePasswordRealm realm = new CachingUsernamePasswordRealm(config, threadPool) {
             @Override
@@ -478,7 +487,11 @@ public class CachingUsernamePasswordRealmTests extends ESTestCase {
         final AtomicInteger authCounter = new AtomicInteger(0);
         final Hasher pwdHasher = Hasher.resolve(randomFrom("pbkdf2", "pbkdf2_1000", "bcrypt", "bcrypt9"));
         final String passwordHash = new String(pwdHasher.hash(password));
-        RealmConfig config = new RealmConfig(new RealmConfig.RealmIdentifier("caching", "test_realm"), globalSettings,
+        final RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("caching", "test_realm");
+        RealmConfig config = new RealmConfig(
+            realmIdentifier,
+            Settings.builder().put(globalSettings)
+                .put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build(),
             TestEnvironment.newEnvironment(globalSettings), new ThreadContext(Settings.EMPTY));
 
         final int numberOfProcessors = Runtime.getRuntime().availableProcessors();
@@ -561,7 +574,10 @@ public class CachingUsernamePasswordRealmTests extends ESTestCase {
         final SecureString randomPassword = new SecureString(randomAlphaOfLength(password.length()).toCharArray());
         final Hasher localHasher = Hasher.resolve(randomFrom("pbkdf2", "pbkdf2_1000", "bcrypt", "bcrypt9"));
         final String passwordHash = new String(localHasher.hash(password));
-        RealmConfig config = new RealmConfig(new RealmConfig.RealmIdentifier("caching", "test_realm"), globalSettings,
+        final RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("caching", "test_realm");
+        RealmConfig config = new RealmConfig(realmIdentifier,
+                Settings.builder().put(globalSettings)
+                    .put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build(),
                 TestEnvironment.newEnvironment(globalSettings), new ThreadContext(Settings.EMPTY));
         final CachingUsernamePasswordRealm realm = new CachingUsernamePasswordRealm(config, threadPool) {
             @Override
@@ -629,8 +645,11 @@ public class CachingUsernamePasswordRealmTests extends ESTestCase {
         final String username = "username";
         final AtomicInteger lookupCounter = new AtomicInteger(0);
 
-        RealmConfig config = new RealmConfig(new RealmConfig.RealmIdentifier("caching", "test_realm"), globalSettings,
-                TestEnvironment.newEnvironment(globalSettings), new ThreadContext(Settings.EMPTY));
+        final RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("caching", "test_realm");
+        RealmConfig config = new RealmConfig(realmIdentifier,
+            Settings.builder().put(globalSettings)
+                .put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build(),
+            TestEnvironment.newEnvironment(globalSettings), new ThreadContext(Settings.EMPTY));
         final CachingUsernamePasswordRealm realm = new CachingUsernamePasswordRealm(config, threadPool) {
             @Override
             protected void doAuthenticate(UsernamePasswordToken token, ActionListener<AuthenticationResult> listener) {
@@ -688,8 +707,9 @@ public class CachingUsernamePasswordRealmTests extends ESTestCase {
     public void testAuthenticateDisabled() throws Exception {
         final RealmConfig.RealmIdentifier realmId = new RealmConfig.RealmIdentifier("caching", "test_authentication_disabled");
         final Settings settings = Settings.builder()
-            .put(RealmSettings.getFullSettingKey(realmId, CachingUsernamePasswordRealmSettings.AUTHC_ENABLED_SETTING), false)
+            .put(getFullSettingKey(realmId, CachingUsernamePasswordRealmSettings.AUTHC_ENABLED_SETTING), false)
             .put(globalSettings)
+            .put(getFullSettingKey(realmId, RealmSettings.ORDER_SETTING), 0)
             .build();
         final Environment env = TestEnvironment.newEnvironment(settings);
         final ThreadContext threadContext = new ThreadContext(settings);
@@ -717,9 +737,16 @@ public class CachingUsernamePasswordRealmTests extends ESTestCase {
     static class FailingAuthenticationRealm extends CachingUsernamePasswordRealm {
 
         FailingAuthenticationRealm(Settings global, ThreadPool threadPool) {
-            super(new RealmConfig(new RealmConfig.RealmIdentifier("caching", "failing-test"), global,
-                    TestEnvironment.newEnvironment(global),
-                    threadPool.getThreadContext()), threadPool);
+            super(new RealmConfig(
+                new RealmConfig.RealmIdentifier("caching", "failing-test"),
+                Settings.builder()
+                    .put(global)
+                    .put(getFullSettingKey(
+                        new RealmConfig.RealmIdentifier("caching", "failing-test"),
+                        RealmSettings.ORDER_SETTING), 0)
+                    .build(),
+                TestEnvironment.newEnvironment(global),
+                threadPool.getThreadContext()), threadPool);
         }
 
         @Override
@@ -736,9 +763,14 @@ public class CachingUsernamePasswordRealmTests extends ESTestCase {
     static class ThrowingAuthenticationRealm extends CachingUsernamePasswordRealm {
 
         ThrowingAuthenticationRealm(Settings globalSettings, ThreadPool threadPool) {
-            super(new RealmConfig(new RealmConfig.RealmIdentifier("caching", "throwing-test"), globalSettings,
-                    TestEnvironment.newEnvironment(globalSettings),
-                    threadPool.getThreadContext()), threadPool);
+            super(new RealmConfig(
+                new RealmConfig.RealmIdentifier("caching", "throwing-test"),
+                Settings.builder()
+                    .put(globalSettings)
+                    .put(getFullSettingKey(new RealmConfig.RealmIdentifier("caching", "throwing-test"), RealmSettings.ORDER_SETTING), 0)
+                    .build(),
+                TestEnvironment.newEnvironment(globalSettings),
+                threadPool.getThreadContext()), threadPool);
         }
 
         @Override
@@ -760,9 +792,14 @@ public class CachingUsernamePasswordRealmTests extends ESTestCase {
         private boolean usersEnabled = true;
 
         AlwaysAuthenticateCachingRealm(Settings globalSettings, ThreadPool threadPool) {
-            this(new RealmConfig(new RealmConfig.RealmIdentifier("caching", "always-test"), globalSettings,
-                    TestEnvironment.newEnvironment(globalSettings),
-                    threadPool.getThreadContext()), threadPool);
+            this(new RealmConfig(
+                new RealmConfig.RealmIdentifier("caching", "always-test"),
+                Settings.builder()
+                    .put(globalSettings)
+                    .put(getFullSettingKey(new RealmConfig.RealmIdentifier("caching", "always-test"), RealmSettings.ORDER_SETTING), 0)
+                    .build(),
+                TestEnvironment.newEnvironment(globalSettings),
+                threadPool.getThreadContext()), threadPool);
         }
 
         AlwaysAuthenticateCachingRealm(RealmConfig config, ThreadPool threadPool) {

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/support/DelegatedAuthorizationSupportTests.java

@@ -16,6 +16,7 @@ import org.elasticsearch.test.ESTestCase;
 import org.elasticsearch.xpack.core.security.authc.AuthenticationResult;
 import org.elasticsearch.xpack.core.security.authc.Realm;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.user.User;
 import org.junit.Before;
 
@@ -63,10 +64,13 @@ public class DelegatedAuthorizationSupportTests extends ESTestCase {
     }
 
     private RealmConfig buildRealmConfig(String name, Settings settings) {
-        return new RealmConfig(new RealmConfig.RealmIdentifier("test", name),
+        RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("test", name);
+        return new RealmConfig(
+            realmIdentifier,
             Settings.builder().put(settings)
                 .normalizePrefix("xpack.security.authc.realms.test." + name + ".")
                 .put(globalSettings)
+                .put(RealmSettings.getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0)
                 .build(),
             env, threadContext);
     }

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/support/DnRoleMapperTests.java

@@ -19,6 +19,7 @@ import org.elasticsearch.threadpool.ThreadPool;
 import org.elasticsearch.watcher.ResourceWatcherService;
 import org.elasticsearch.xpack.core.security.audit.logfile.CapturingLogger;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.support.DnRoleMapperSettings;
 import org.junit.After;
 import org.junit.Before;
@@ -296,6 +297,7 @@ public class DnRoleMapperTests extends ESTestCase {
         Settings ldapSettings = Settings.builder()
             .put(settings)
             .put(getFullSettingKey(realmIdentifier, DnRoleMapperSettings.ROLE_MAPPING_FILE_SETTING), file.toAbsolutePath())
+            .put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0)
             .build();
         RealmConfig config = new RealmConfig(realmIdentifier, ldapSettings,
                 TestEnvironment.newEnvironment(settings), new ThreadContext(Settings.EMPTY));
@@ -313,6 +315,7 @@ public class DnRoleMapperTests extends ESTestCase {
         Settings ldapSettings = Settings.builder()
                 .put(settings)
                 .put(getFullSettingKey(realmIdentifier, DnRoleMapperSettings.USE_UNMAPPED_GROUPS_AS_ROLES_SETTING), true)
+                .put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0)
                 .build();
         RealmConfig config = new RealmConfig(realmIdentifier, ldapSettings,
                 TestEnvironment.newEnvironment(settings), new ThreadContext(Settings.EMPTY));
@@ -330,6 +333,7 @@ public class DnRoleMapperTests extends ESTestCase {
                 .put(settings)
                 .put(getFullSettingKey(realmIdentifier, DnRoleMapperSettings.ROLE_MAPPING_FILE_SETTING), file.toAbsolutePath())
                 .put(getFullSettingKey(realmIdentifier, DnRoleMapperSettings.USE_UNMAPPED_GROUPS_AS_ROLES_SETTING), false)
+                .put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0)
                 .build();
         RealmConfig config = new RealmConfig(realmIdentifier, ldapSettings,
                 TestEnvironment.newEnvironment(settings), new ThreadContext(Settings.EMPTY));
@@ -345,6 +349,7 @@ public class DnRoleMapperTests extends ESTestCase {
         Settings mergedSettings = Settings.builder()
                 .put(settings)
                 .put(getFullSettingKey(identifier, DnRoleMapperSettings.ROLE_MAPPING_FILE_SETTING), file.toAbsolutePath())
+                .put(getFullSettingKey(identifier, RealmSettings.ORDER_SETTING), 0)
                 .build();
         RealmConfig config = new RealmConfig(identifier, mergedSettings, env, new ThreadContext(Settings.EMPTY));
         return new DnRoleMapper(config, watcherService);

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/support/RealmUserLookupTests.java

@@ -19,6 +19,7 @@ import org.elasticsearch.xpack.core.security.authc.AuthenticationToken;
 import org.elasticsearch.xpack.core.security.authc.Realm;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig.RealmIdentifier;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.user.User;
 import org.junit.Before;
 
@@ -85,7 +86,11 @@ public class RealmUserLookupTests extends ESTestCase {
     }
 
     public void testRealmException() {
-        final Realm realm = new Realm(new RealmConfig(new RealmIdentifier("test", "test"), globalSettings, env, threadContext)) {
+        RealmIdentifier realmIdentifier = new RealmIdentifier("test", "test");
+        final Realm realm = new Realm(new RealmConfig(realmIdentifier,
+            Settings.builder().put(globalSettings)
+                .put(RealmSettings.getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build(),
+            env, threadContext)) {
             @Override
             public boolean supports(AuthenticationToken token) {
                 return false;
@@ -116,7 +121,12 @@ public class RealmUserLookupTests extends ESTestCase {
     private List<MockLookupRealm> buildRealms(int realmCount) {
         final List<MockLookupRealm> realms = new ArrayList<>(realmCount);
         for (int i = 1; i <= realmCount; i++) {
-            final RealmConfig config = new RealmConfig(new RealmIdentifier("mock","lookup-" + i), globalSettings, env, threadContext);
+            RealmIdentifier realmIdentifier = new RealmIdentifier("mock", "lookup-" + i);
+            final RealmConfig config = new RealmConfig(realmIdentifier,
+                Settings.builder().put(globalSettings)
+                    .put(RealmSettings.getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build(),
+                env,
+                threadContext);
             final MockLookupRealm realm = new MockLookupRealm(config);
             for (int j = 0; j < 5; j++) {
                 realm.registerUser(new User(randomAlphaOfLengthBetween(6, 12)));

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/support/RoleMappingFileBootstrapCheckTests.java

@@ -55,7 +55,9 @@ public class RoleMappingFileBootstrapCheckTests extends AbstractBootstrapCheckTe
     }
 
     private static RealmConfig getRealmConfig(Settings settings) {
-        return new RealmConfig(REALM_ID, settings, TestEnvironment.newEnvironment(settings), new ThreadContext(Settings.EMPTY));
+        return new RealmConfig(REALM_ID,
+            Settings.builder().put(settings).put(RealmSettings.getFullSettingKey(REALM_ID, RealmSettings.ORDER_SETTING), 0).build(),
+            TestEnvironment.newEnvironment(settings), new ThreadContext(Settings.EMPTY));
     }
 
     public void testBootstrapCheckOfMissingFile() {

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/support/mapper/ExpressionRoleMappingTests.java

@@ -28,6 +28,7 @@ import org.elasticsearch.test.ESTestCase;
 import org.elasticsearch.test.VersionUtils;
 import org.elasticsearch.xpack.core.XPackClientPlugin;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.support.mapper.ExpressionRoleMapping;
 import org.elasticsearch.xpack.core.security.authc.support.mapper.TemplateRoleName;
 import org.elasticsearch.xpack.core.security.authc.support.mapper.expressiondsl.AllExpression;
@@ -57,8 +58,11 @@ public class ExpressionRoleMappingTests extends ESTestCase {
 
     @Before
     public void setupMapping() throws Exception {
-        realm = new RealmConfig(new RealmConfig.RealmIdentifier("ldap", "ldap1"),
-            Settings.EMPTY, Mockito.mock(Environment.class), new ThreadContext(Settings.EMPTY));
+        RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("ldap", "ldap1");
+        realm = new RealmConfig(
+            realmIdentifier,
+            Settings.builder().put(RealmSettings.getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build(),
+            Mockito.mock(Environment.class), new ThreadContext(Settings.EMPTY));
     }
 
     public void testValidExpressionWithFixedRoleNames() throws Exception {

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/support/mapper/NativeRoleMappingStoreTests.java

@@ -26,6 +26,7 @@ import org.elasticsearch.xpack.core.security.action.realm.ClearRealmCacheRequest
 import org.elasticsearch.xpack.core.security.action.realm.ClearRealmCacheResponse;
 import org.elasticsearch.xpack.core.security.authc.AuthenticationResult;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken;
 import org.elasticsearch.xpack.core.security.authc.support.mapper.ExpressionRoleMapping;
 import org.elasticsearch.xpack.core.security.authc.support.mapper.TemplateRoleName;
@@ -100,8 +101,11 @@ public class NativeRoleMappingStoreTests extends ESTestCase {
             }
         };
 
-        final RealmConfig realm = new RealmConfig(new RealmConfig.RealmIdentifier("ldap", "ldap1"), Settings.EMPTY,
-                mock(Environment.class), new ThreadContext(Settings.EMPTY));
+        RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("ldap", "ldap1");
+        final Settings settings = Settings.builder()
+            .put(RealmSettings.getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build();
+        final RealmConfig realm = new RealmConfig(realmIdentifier, settings,
+                mock(Environment.class), new ThreadContext(settings));
 
         final PlainActionFuture<Set<String>> future = new PlainActionFuture<>();
         final UserRoleMapper.UserData user = new UserRoleMapper.UserData("sasquatch",
@@ -235,7 +239,10 @@ public class NativeRoleMappingStoreTests extends ESTestCase {
         if (attachRealm) {
             final Environment env = TestEnvironment.newEnvironment(settings);
             final RealmConfig.RealmIdentifier identifier = new RealmConfig.RealmIdentifier("ldap", realmName);
-            final RealmConfig realmConfig = new RealmConfig(identifier, settings, env, threadContext);
+            final RealmConfig realmConfig = new RealmConfig(identifier,
+                Settings.builder().put(settings)
+                    .put(RealmSettings.getFullSettingKey(identifier, RealmSettings.ORDER_SETTING), 0).build(),
+                env, threadContext);
             final CachingUsernamePasswordRealm mockRealm = new CachingUsernamePasswordRealm(realmConfig, threadPool) {
                 @Override
                 protected void doAuthenticate(UsernamePasswordToken token, ActionListener<AuthenticationResult> listener) {

x-pack/qa/openldap-tests/src/test/java/org/elasticsearch/test/OpenLdapTests.java

@@ -19,6 +19,7 @@ import org.elasticsearch.env.TestEnvironment;
 import org.elasticsearch.threadpool.TestThreadPool;
 import org.elasticsearch.threadpool.ThreadPool;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.ldap.SearchGroupsResolverSettings;
 import org.elasticsearch.xpack.core.security.authc.ldap.support.LdapMetaDataResolverSettings;
 import org.elasticsearch.xpack.core.security.authc.ldap.support.LdapSearchScope;
@@ -209,6 +210,7 @@ public class OpenLdapTests extends ESTestCase {
         final Settings settings = Settings.builder()
                 .putList(getFullSettingKey(realmId.getName(), LdapMetaDataResolverSettings.ADDITIONAL_META_DATA_SETTING.apply("ldap")),
                         "cn", "sn")
+                .put(getFullSettingKey(realmId, RealmSettings.ORDER_SETTING), 0)
                 .build();
         final RealmConfig config = new RealmConfig(realmId, settings,
                 TestEnvironment.newEnvironment(globalSettings), new ThreadContext(Settings.EMPTY));
@@ -226,6 +228,7 @@ public class OpenLdapTests extends ESTestCase {
         final Settings settings = Settings.builder()
                 .putList(getFullSettingKey(realmId.getName(), LdapMetaDataResolverSettings.ADDITIONAL_META_DATA_SETTING.apply("ldap")),
                         "objectClass")
+                .put(getFullSettingKey(realmId, RealmSettings.ORDER_SETTING), 0)
                 .build();
         final RealmConfig config = new RealmConfig(realmId, settings,
                 TestEnvironment.newEnvironment(globalSettings), new ThreadContext(Settings.EMPTY));
@@ -243,6 +246,7 @@ public class OpenLdapTests extends ESTestCase {
         final Settings settings = Settings.builder()
                 .putList(getFullSettingKey(realmId.getName(), LdapMetaDataResolverSettings.ADDITIONAL_META_DATA_SETTING.apply("ldap")),
                         "alias")
+                .put(getFullSettingKey(realmId, RealmSettings.ORDER_SETTING), 0)
                 .build();
         final RealmConfig config = new RealmConfig(realmId, settings,
                 TestEnvironment.newEnvironment(globalSettings), new ThreadContext(Settings.EMPTY));
@@ -264,6 +268,7 @@ public class OpenLdapTests extends ESTestCase {
             .put(getFullSettingKey(realmId, SSLConfigurationSettings.TRUST_STORE_PATH_REALM), getDataPath(LDAPTRUST_PATH))
             .put(getFullSettingKey(realmId, SSLConfigurationSettings.LEGACY_TRUST_STORE_PASSWORD_REALM), "changeit")
             .put(globalSettings)
+            .put(getFullSettingKey(realmId, RealmSettings.ORDER_SETTING), 0)
             .build();
     }
 

x-pack/qa/openldap-tests/src/test/java/org/elasticsearch/xpack/security/authc/ldap/OpenLdapUserSearchSessionFactoryTests.java

@@ -18,6 +18,7 @@ import org.elasticsearch.test.OpenLdapTests;
 import org.elasticsearch.threadpool.TestThreadPool;
 import org.elasticsearch.threadpool.ThreadPool;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.ldap.LdapUserSearchSessionFactorySettings;
 import org.elasticsearch.xpack.core.security.authc.ldap.PoolingSessionFactorySettings;
 import org.elasticsearch.xpack.core.security.authc.ldap.SearchGroupsResolverSettings;
@@ -90,7 +91,8 @@ public class OpenLdapUserSearchSessionFactoryTests extends ESTestCase {
         } else {
             realmSettings.put(getFullSettingKey(realmId, PoolingSessionFactorySettings.LEGACY_BIND_PASSWORD), OpenLdapTests.PASSWORD);
         }
-        final Settings settings = realmSettings.put(globalSettings).build();
+        final Settings settings = realmSettings.put(globalSettings)
+            .put(getFullSettingKey(realmId, RealmSettings.ORDER_SETTING), 0).build();
         RealmConfig config = new RealmConfig(realmId, settings,
                 TestEnvironment.newEnvironment(globalSettings), new ThreadContext(globalSettings));
 

x-pack/qa/security-example-spi-extension/src/test/java/org/elasticsearch/example/realm/CustomRealmTests.java

@@ -13,17 +13,22 @@ import org.elasticsearch.env.TestEnvironment;
 import org.elasticsearch.test.ESTestCase;
 import org.elasticsearch.xpack.core.security.authc.AuthenticationResult;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken;
 import org.elasticsearch.xpack.core.security.user.User;
 
+import static org.elasticsearch.xpack.core.security.authc.RealmSettings.getFullSettingKey;
 import static org.hamcrest.Matchers.equalTo;
 import static org.hamcrest.Matchers.notNullValue;
 
 public class CustomRealmTests extends ESTestCase {
     public void testAuthenticate() {
         Settings globalSettings = Settings.builder().put("path.home", createTempDir()).build();
-        CustomRealm realm = new CustomRealm(new RealmConfig(new RealmConfig.RealmIdentifier(CustomRealm.TYPE, "test"),
-                globalSettings, TestEnvironment.newEnvironment(globalSettings), new ThreadContext(globalSettings)));
+        final RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier(CustomRealm.TYPE, "test");
+        CustomRealm realm = new CustomRealm(new RealmConfig(
+            realmIdentifier,
+            Settings.builder().put(globalSettings).put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build(),
+            TestEnvironment.newEnvironment(globalSettings), new ThreadContext(globalSettings)));
         SecureString password = CustomRealm.KNOWN_PW.clone();
         UsernamePasswordToken token = new UsernamePasswordToken(CustomRealm.KNOWN_USER, password);
         PlainActionFuture<AuthenticationResult> plainActionFuture = new PlainActionFuture<>();
@@ -36,8 +41,11 @@ public class CustomRealmTests extends ESTestCase {
 
     public void testAuthenticateBadUser() {
         Settings globalSettings = Settings.builder().put("path.home", createTempDir()).build();
-        CustomRealm realm = new CustomRealm(new RealmConfig(new RealmConfig.RealmIdentifier(CustomRealm.TYPE, "test"),
-                globalSettings, TestEnvironment.newEnvironment(globalSettings), new ThreadContext(globalSettings)));
+        final RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier(CustomRealm.TYPE, "test");
+        CustomRealm realm = new CustomRealm(new RealmConfig(
+            realmIdentifier,
+            Settings.builder().put(globalSettings).put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build(),
+            TestEnvironment.newEnvironment(globalSettings), new ThreadContext(globalSettings)));
         SecureString password = CustomRealm.KNOWN_PW.clone();
         UsernamePasswordToken token = new UsernamePasswordToken(CustomRealm.KNOWN_USER + "1", password);
         PlainActionFuture<AuthenticationResult> plainActionFuture = new PlainActionFuture<>();

x-pack/qa/security-example-spi-extension/src/test/java/org/elasticsearch/example/realm/CustomRoleMappingRealmTests.java

@@ -7,10 +7,12 @@ package org.elasticsearch.example.realm;
 
 import org.elasticsearch.action.ActionListener;
 import org.elasticsearch.action.support.PlainActionFuture;
+import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.common.util.concurrent.ThreadContext;
 import org.elasticsearch.env.Environment;
 import org.elasticsearch.test.ESTestCase;
 import org.elasticsearch.xpack.core.security.authc.RealmConfig;
+import org.elasticsearch.xpack.core.security.authc.RealmSettings;
 import org.elasticsearch.xpack.core.security.authc.support.UserRoleMapper;
 import org.elasticsearch.xpack.core.security.user.User;
 
@@ -18,6 +20,7 @@ import java.util.Set;
 import java.util.concurrent.atomic.AtomicInteger;
 import java.util.function.Supplier;
 
+import static org.elasticsearch.xpack.core.security.authc.RealmSettings.getFullSettingKey;
 import static org.hamcrest.Matchers.arrayContainingInAnyOrder;
 import static org.hamcrest.Matchers.is;
 import static org.hamcrest.Matchers.sameInstance;
@@ -30,9 +33,11 @@ public class CustomRoleMappingRealmTests extends ESTestCase {
     public void testCachingOfUserLookup() throws Exception {
         final Environment env = super.newEnvironment();
         final UserRoleMapper roleMapper = mock(UserRoleMapper.class);
+        final RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier(CustomRoleMappingRealm.TYPE, "test");
         final RealmConfig realmConfig = new RealmConfig(
-            new RealmConfig.RealmIdentifier(CustomRoleMappingRealm.TYPE, "test"),
-            env.settings(), env, new ThreadContext(env.settings())
+            realmIdentifier,
+            Settings.builder().put(env.settings()).put(getFullSettingKey(realmIdentifier, RealmSettings.ORDER_SETTING), 0).build(),
+            env, new ThreadContext(env.settings())
         );
         CustomRoleMappingRealm realm = new CustomRoleMappingRealm(realmConfig, roleMapper);
 

x-pack/qa/third-party/active-directory/src/test/java/org/elasticsearch/xpack/security/authc/ldap/ADLdapUserSearchSessionFactoryTests.java

@@ -58,6 +58,7 @@ public class ADLdapUserSearchSessionFactoryTests extends AbstractActiveDirectory
     }
 
     public void testUserSearchWithActiveDirectory() throws Exception {
+        final RealmConfig.RealmIdentifier realmIdentifier = new RealmConfig.RealmIdentifier("ldap", "ad-as-ldap-test");
         String groupSearchBase = "DC=ad,DC=test,DC=elasticsearch,DC=com";
         String userSearchBase = "CN=Users,DC=ad,DC=test,DC=elasticsearch,DC=com";
         Settings settings = Settings.builder()
@@ -69,16 +70,16 @@ public class ADLdapUserSearchSessionFactoryTests extends AbstractActiveDirectory
                 .put("user_search.filter", "(cn={0})")
                 .put("user_search.pool.enabled", randomBoolean())
                 .put("follow_referrals", ActiveDirectorySessionFactoryTests.FOLLOW_REFERRALS)
+                .put("order", 0)
                 .build();
         Settings.Builder builder = Settings.builder()
                 .put(globalSettings);
         settings.keySet().forEach(k -> {
             builder.copy("xpack.security.authc.realms.ldap.ad-as-ldap-test." + k, k, settings);
-
         });
         Settings fullSettings = builder.build();
         sslService = new SSLService(TestEnvironment.newEnvironment(fullSettings));
-        RealmConfig config = new RealmConfig(new RealmConfig.RealmIdentifier("ldap", "ad-as-ldap-test"), fullSettings,
+        RealmConfig config = new RealmConfig(realmIdentifier, fullSettings,
                 TestEnvironment.newEnvironment(fullSettings), new ThreadContext(fullSettings));
         LdapUserSearchSessionFactory sessionFactory = getLdapUserSearchSessionFactory(config, sslService, threadPool);
 

x-pack/qa/third-party/active-directory/src/test/java/org/elasticsearch/xpack/security/authc/ldap/AbstractAdLdapRealmTestCase.java

@@ -381,7 +381,7 @@ public abstract class AbstractAdLdapRealmTestCase extends SecurityIntegTestCase
         }
 
         public Settings buildSettings(List<String> certificateAuthorities) {
-            return buildSettings(certificateAuthorities, 1);
+            return buildSettings(certificateAuthorities, randomInt());
         }
 
 

x-pack/qa/third-party/active-directory/src/test/java/org/elasticsearch/xpack/security/authc/ldap/ActiveDirectorySessionFactoryTests.java

@@ -87,14 +87,15 @@ public class ActiveDirectorySessionFactoryTests extends AbstractActiveDirectoryT
     }
 
     private RealmConfig configureRealm(String name, String type, Settings settings) {
+        final RealmConfig.RealmIdentifier identifier = new RealmConfig.RealmIdentifier(type, name);
         final Settings mergedSettings = Settings.builder()
             .put(settings)
             .normalizePrefix("xpack.security.authc.realms." + type + "." + name + ".")
             .put(globalSettings)
+            .put(getFullSettingKey(identifier, RealmSettings.ORDER_SETTING), 0)
             .build();
         final Environment env = TestEnvironment.newEnvironment(mergedSettings);
         this.sslService = new SSLService(env);
-        final RealmConfig.RealmIdentifier identifier = new RealmConfig.RealmIdentifier(type, name);
         return new RealmConfig(identifier, mergedSettings, env, new ThreadContext(globalSettings));
     }