Remove HTTPS check for API Keys & Service Accounts (#76801)

This commit removes the checks that prevented the use of API Keys and
Service Account (Service Tokens) on nodes without HTTPS
(xpack.security.http.ssl.enabled)

As a consequence of removing this check, the API Key service is now
automatically enabled, but can be explicitly disabled with

     xpack.security.authc.api_key.enabled: false

docs/reference/settings/security-settings.asciidoc

@@ -178,9 +178,7 @@ You can set the following API key service settings in
 
 `xpack.security.authc.api_key.enabled`::
 (<<static-cluster-setting,Static>>)
-Set to `false` to disable the built-in API key service. Defaults to `true` unless
- `xpack.security.http.ssl.enabled` is `false`. This prevents sniffing the API key
-  from a connection over plain http.
+Set to `false` to disable the built-in API key service. Defaults to `true`.
 
 `xpack.security.authc.api_key.hashing.algorithm`::
 (<<static-cluster-setting,Static>>)

x-pack/docs/en/rest-api/security/create-api-keys.asciidoc

@@ -26,11 +26,8 @@ See the note under <<api-key-role-descriptors,`role_descriptors`>>.
 [[security-api-create-api-key-desc]]
 ==== {api-description-title}
 
-The API keys are created by the {es} API key service, which is automatically enabled
-when you <<encrypt-http-communication,configure TLS on the HTTP interface>>. Alternatively,
-you can explicitly enable the `xpack.security.authc.api_key.enabled` setting. When
-you are running in production mode, a bootstrap check prevents you from enabling
-the API key service unless you also enable TLS on the HTTP interface.
+The API keys are created by the {es} API key service, which is automatically enabled.
+For instructions on disabling the API key service, see <<api-key-service-settings>>.
 
 A successful request returns a JSON structure that contains the
 API key, its unique id, and its name. If applicable, it also returns expiration

x-pack/docs/en/rest-api/security/create-service-token.asciidoc

@@ -25,7 +25,6 @@ authentication.
 
 [[security-api-create-service-token-desc]]
 ==== {api-description-title}
-include::../../security/authentication/service-accounts.asciidoc[tag=service-accounts-tls]
 
 A successful create service account token API call returns a JSON structure
 that contains the service account token, its name, and its secret value.

x-pack/docs/en/rest-api/security/delete-service-token.asciidoc

@@ -21,7 +21,6 @@ specified `namespace`.
 
 [[security-api-delete-service-token-desc]]
 ==== {api-description-title}
-include::../../security/authentication/service-accounts.asciidoc[tag=service-accounts-tls]
 
 The API response indicates whether the specified service account token is found
 and deleted or it is not found.

x-pack/docs/en/rest-api/security/get-service-accounts.asciidoc

@@ -28,7 +28,7 @@ NOTE: Currently, only the `elastic/fleet-server` service account is available.
 [[security-api-get-service-accounts-desc]]
 ==== {api-description-title}
 
-include::../../security/authentication/service-accounts.asciidoc[tag=service-accounts-tls]
+This API returns a list of service accounts that match the provided path parameter(s).
 
 [[security-api-get-service-accounts-path-params]]
 ==== {api-path-parms-title}

x-pack/docs/en/rest-api/security/get-service-credentials.asciidoc

@@ -21,7 +21,6 @@ Retrieves all service credentials for a  <<service-accounts,service account>>.
 
 [[security-api-get-service-credentials-desc]]
 ==== {api-description-title}
-include::../../security/authentication/service-accounts.asciidoc[tag=service-accounts-tls]
 
 Use this API to retrieve a list of credentials for a service account.
 The response includes service account tokens that were created with the

x-pack/docs/en/rest-api/security/grant-api-keys.asciidoc

@@ -23,15 +23,16 @@ Creates an API key on behalf of another user.
 This API is similar to <<security-api-create-api-key>>, however it creates the
 API key for a user that is different than the user that runs the API.
 
-The caller must have authentication credentials (either an access token, or a username and password) for the user on whose behalf the API key will be created. It is not possible to use this API to create an API key without that user's credentials.
-
-This API is intended be used by applications that need to create and manage API keys for end users, but cannot guarantee that those users have permission to create API keys on their own behalf (see <<security-api-create-api-key-prereqs>>).
+The caller must have authentication credentials (either an access token,
+or a username and password) for the user on whose behalf the API key will be
+created. It is not possible to use this API to create an API key without that
+user's credentials.
+
+This API is intended be used by applications that need to create and manage
+API keys for end users, but cannot guarantee that those users have permission
+to create API keys on their own behalf (see <<security-api-create-api-key-prereqs>>).
 The API keys are created by the {es} API key service, which is automatically
-enabled when you configure TLS on the HTTP interface. See <<encrypt-http-communication>>.
-Alternatively, you can explicitly enable the
-`xpack.security.authc.api_key.enabled` setting. When you are running in
-production mode, a bootstrap check prevents you from enabling the API key
-service unless you also enable TLS on the HTTP interface.
+enabled.
 
 A successful grant API key API call returns a JSON structure that contains the
 API key, its unique id, and its name. If applicable, it also returns expiration

x-pack/docs/en/security/authentication/overview.asciidoc

@@ -25,10 +25,12 @@ you must attach your user credentials to the requests sent to {es}. For example,
 when using realms that support usernames and passwords you can simply attach
 {wikipedia}/Basic_access_authentication[basic auth] header to the requests.
 
-The {security-features} provide two services: the token service and the api key
+The {security-features} provide two services: the token service and the API key
 service. You can use these services to exchange the current authentication for
 a token or key. This token or key can then be used as credentials for authenticating
-new requests. These services are enabled by default when TLS/SSL is enabled for HTTP.
+new requests.
+The API key service is enabled by default.
+The token service is enabled by default when TLS/SSL is enabled for HTTP.
 
 include::built-in-users.asciidoc[][]
 include::service-accounts.asciidoc[]

x-pack/docs/en/security/authentication/service-accounts.asciidoc

@@ -57,13 +57,6 @@ users. Service accounts can only be authenticated with service tokens, which are
 not applicable to regular users.
 // end::service-accounts-usage[]
 
-// tag::service-accounts-tls[]
-In <<dev-vs-prod-mode,production mode>>, service accounts require TLS on the
-HTTP interface. A runtime check prevents you from invoking any related APIs or
-authenticating with a service account token unless TLS is enabled on the HTTP
-interface. See <<encrypt-http-communication,encrypt HTTP client communications for {es}>>.
-// end::service-accounts-tls[]
-
 [discrete]
 [[service-accounts-tokens]]
 === Service account tokens

x-pack/docs/en/security/securing-communications/security-basic-setup-https.asciidoc

@@ -4,10 +4,9 @@
 <titleabbrev>Set up basic security plus HTTPS</titleabbrev>
 ++++
 
-In a production environment, some {es} features such as tokens and
-API keys will be disabled unless you enable TLS on the HTTP layer. This
-additional layer of security ensures that all communications to and from your
-cluster are secured.
+When you enable TLS on the HTTP layer it provides an additional layer of
+security to ensure that all communications to and from your
+cluster are encrypted.
 
 When you run the `elasticsearch-certutil` tool in `http` mode, the tool asks
 several questions about how you want to generate certificates. While there are

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/XPackSettings.java

@@ -93,9 +93,9 @@ public class XPackSettings {
     public static final Setting<Boolean> TOKEN_SERVICE_ENABLED_SETTING =
         Setting.boolSetting("xpack.security.authc.token.enabled", XPackSettings.HTTP_SSL_ENABLED, Setting.Property.NodeScope);
 
-    /** Setting for enabling or disabling the api key service. Defaults to the value of https being enabled */
+    /** Setting for enabling or disabling the api key service. Defaults to true */
     public static final Setting<Boolean> API_KEY_SERVICE_ENABLED_SETTING =
-        Setting.boolSetting("xpack.security.authc.api_key.enabled", XPackSettings.HTTP_SSL_ENABLED, Setting.Property.NodeScope);
+        Setting.boolSetting("xpack.security.authc.api_key.enabled", true, Setting.Property.NodeScope);
 
     /** Setting for enabling or disabling FIPS mode. Defaults to false */
     public static final Setting<Boolean> FIPS_MODE_ENABLED =

x-pack/plugin/security/src/internalClusterTest/java/org/elasticsearch/license/LicensingTests.java

@@ -20,6 +20,7 @@ import org.elasticsearch.client.Client;
 import org.elasticsearch.client.Request;
 import org.elasticsearch.client.RequestOptions;
 import org.elasticsearch.client.ResponseException;
+import org.elasticsearch.common.Strings;
 import org.elasticsearch.common.settings.SecureString;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.common.util.CollectionUtils;
@@ -47,7 +48,9 @@ import java.nio.file.Path;
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Collection;
+import java.util.Comparator;
 import java.util.List;
+import java.util.Locale;
 import java.util.concurrent.TimeUnit;
 import java.util.stream.Collectors;
 
@@ -56,6 +59,9 @@ import static org.elasticsearch.discovery.SettingsBasedSeedHostsProvider.DISCOVE
 import static org.elasticsearch.license.LicenseService.LICENSE_EXPIRATION_WARNING_PERIOD;
 import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertNoFailures;
 import static org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken.basicAuthHeaderValue;
+import static org.hamcrest.Matchers.arrayWithSize;
+import static org.hamcrest.Matchers.containsStringIgnoringCase;
+import static org.hamcrest.Matchers.equalToIgnoringCase;
 import static org.hamcrest.Matchers.greaterThanOrEqualTo;
 import static org.hamcrest.Matchers.hasItem;
 import static org.hamcrest.Matchers.is;
@@ -224,10 +230,19 @@ public class LicensingTests extends SecurityIntegTestCase {
             getRestClient().performRequest(request);
         } catch (ResponseException e) {
             headers = e.getResponse().getHeaders();
-            List<String> afterWarningHeaders= getWarningHeaders(e.getResponse().getHeaders());
+            List<String> afterWarningHeaders = getWarningHeaders(headers);
             assertThat(afterWarningHeaders, Matchers.hasSize(0));
         }
-        assertThat(headers != null && headers.length == 3, is(true));
+        assertThat(headers, notNullValue());
+        assertThat(Strings.arrayToCommaDelimitedString(headers), headers, arrayWithSize(4));
+
+        Arrays.sort(headers, Comparator.comparing((Header h) -> h.getName().toLowerCase(Locale.ROOT)).thenComparing(Header::getValue));
+        assertThat(headers[0].getName(), equalToIgnoringCase("content-length"));
+        assertThat(headers[1].getName(), equalToIgnoringCase("content-type"));
+        assertThat(headers[2].getName(), equalToIgnoringCase("WWW-Authenticate"));
+        assertThat(headers[2].getValue(), containsStringIgnoringCase("ApiKey"));
+        assertThat(headers[3].getName(), equalToIgnoringCase("WWW-Authenticate"));
+        assertThat(headers[3].getValue(), containsStringIgnoringCase("Basic"));
     }
 
     private static void assertElasticsearchSecurityException(ThrowingRunnable runnable) {

x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/ApiKeySSLBootstrapCheck.java

@@ -1,38 +0,0 @@
-/*
- * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
- * or more contributor license agreements. Licensed under the Elastic License
- * 2.0; you may not use this file except in compliance with the Elastic License
- * 2.0.
- */
-
-package org.elasticsearch.xpack.security;
-
-import org.elasticsearch.bootstrap.BootstrapCheck;
-import org.elasticsearch.bootstrap.BootstrapContext;
-import org.elasticsearch.xpack.core.XPackSettings;
-
-import java.util.Locale;
-
-/**
- * Bootstrap check to ensure that the user has enabled HTTPS when using the api key service
- */
-public final class ApiKeySSLBootstrapCheck implements BootstrapCheck {
-
-    @Override
-    public BootstrapCheckResult check(BootstrapContext context) {
-        final Boolean httpsEnabled = XPackSettings.HTTP_SSL_ENABLED.get(context.settings());
-        final Boolean apiKeyServiceEnabled = XPackSettings.API_KEY_SERVICE_ENABLED_SETTING.get(context.settings());
-        if (httpsEnabled == false && apiKeyServiceEnabled) {
-            final String message = String.format(
-                    Locale.ROOT,
-                    "HTTPS is required in order to use the API key service; "
-                            + "please enable HTTPS using the [%s] setting or disable the API key service using the [%s] setting",
-                    XPackSettings.HTTP_SSL_ENABLED.getKey(),
-                    XPackSettings.API_KEY_SERVICE_ENABLED_SETTING.getKey());
-            return BootstrapCheckResult.failure(message);
-        }
-        return BootstrapCheckResult.success();
-    }
-
-
-}

x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/Security.java

@@ -227,7 +227,6 @@ import org.elasticsearch.xpack.security.authc.service.CachingServiceAccountToken
 import org.elasticsearch.xpack.security.authc.service.FileServiceAccountTokenStore;
 import org.elasticsearch.xpack.security.authc.service.IndexServiceAccountTokenStore;
 import org.elasticsearch.xpack.security.authc.service.ServiceAccountService;
-import org.elasticsearch.xpack.security.authc.support.HttpTlsRuntimeCheck;
 import org.elasticsearch.xpack.security.authc.support.SecondaryAuthenticator;
 import org.elasticsearch.xpack.security.authc.support.mapper.NativeRoleMappingStore;
 import org.elasticsearch.xpack.security.authz.AuthorizationService;
@@ -459,7 +458,6 @@ public class Security extends Plugin implements SystemIndexPlugin, IngestPlugin,
         // If we wait until #getBoostrapChecks the secure settings will have been cleared/closed.
         final List<BootstrapCheck> checks = new ArrayList<>();
         checks.addAll(Arrays.asList(
-            new ApiKeySSLBootstrapCheck(),
             new TokenSSLBootstrapCheck(),
             new PkiRealmBootstrapCheck(getSslService()),
             new TLSLicenseBootstrapCheck()));
@@ -552,9 +550,6 @@ public class Security extends Plugin implements SystemIndexPlugin, IngestPlugin,
             clusterService, cacheInvalidatorRegistry, threadPool);
         components.add(apiKeyService);
 
-        final HttpTlsRuntimeCheck httpTlsRuntimeCheck = new HttpTlsRuntimeCheck(settings, transportReference);
-        components.add(httpTlsRuntimeCheck);
-
         final IndexServiceAccountTokenStore indexServiceAccountTokenStore = new IndexServiceAccountTokenStore(
             settings, threadPool, getClock(), client, securityIndex.get(), clusterService, cacheInvalidatorRegistry);
         components.add(indexServiceAccountTokenStore);
@@ -563,8 +558,11 @@ public class Security extends Plugin implements SystemIndexPlugin, IngestPlugin,
             new FileServiceAccountTokenStore(environment, resourceWatcherService, threadPool, clusterService, cacheInvalidatorRegistry);
         components.add(fileServiceAccountTokenStore);
 
-        final ServiceAccountService serviceAccountService = new ServiceAccountService(client,
-            fileServiceAccountTokenStore, indexServiceAccountTokenStore, httpTlsRuntimeCheck);
+        final ServiceAccountService serviceAccountService = new ServiceAccountService(
+            client,
+            fileServiceAccountTokenStore,
+            indexServiceAccountTokenStore
+        );
         components.add(serviceAccountService);
 
         final CompositeRolesStore allRolesStore = new CompositeRolesStore(settings, fileRolesStore, nativeRolesStore, reservedRolesStore,

x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/action/service/TransportGetServiceAccountAction.java

@@ -19,38 +19,31 @@ import org.elasticsearch.xpack.core.security.action.service.GetServiceAccountRes
 import org.elasticsearch.xpack.core.security.action.service.ServiceAccountInfo;
 import org.elasticsearch.xpack.security.authc.service.ServiceAccount;
 import org.elasticsearch.xpack.security.authc.service.ServiceAccountService;
-import org.elasticsearch.xpack.security.authc.support.HttpTlsRuntimeCheck;
 
 import java.util.function.Predicate;
 
 public class TransportGetServiceAccountAction extends HandledTransportAction<GetServiceAccountRequest, GetServiceAccountResponse> {
 
-    private final HttpTlsRuntimeCheck httpTlsRuntimeCheck;
-
     @Inject
-    public TransportGetServiceAccountAction(TransportService transportService, ActionFilters actionFilters,
-                                            HttpTlsRuntimeCheck httpTlsRuntimeCheck) {
+    public TransportGetServiceAccountAction(TransportService transportService, ActionFilters actionFilters) {
         super(GetServiceAccountAction.NAME, transportService, actionFilters, GetServiceAccountRequest::new);
-        this.httpTlsRuntimeCheck = httpTlsRuntimeCheck;
     }
 
     @Override
     protected void doExecute(Task task, GetServiceAccountRequest request, ActionListener<GetServiceAccountResponse> listener) {
-        httpTlsRuntimeCheck.checkTlsThenExecute(listener::onFailure, "get service accounts", () -> {
-            Predicate<ServiceAccount> filter = v -> true;
-            if (request.getNamespace() != null) {
-                filter = filter.and( v -> v.id().namespace().equals(request.getNamespace()) );
-            } 
-            if (request.getServiceName() != null) {
-                filter = filter.and( v -> v.id().serviceName().equals(request.getServiceName()) );
-            }
-            final ServiceAccountInfo[] serviceAccountInfos = ServiceAccountService.getServiceAccounts()
-                .values()
-                .stream()
-                .filter(filter)
-                .map(v -> new ServiceAccountInfo(v.id().asPrincipal(), v.roleDescriptor()))
-                .toArray(ServiceAccountInfo[]::new);
-            listener.onResponse(new GetServiceAccountResponse(serviceAccountInfos));
-        });
+        Predicate<ServiceAccount> filter = v -> true;
+        if (request.getNamespace() != null) {
+            filter = filter.and(v -> v.id().namespace().equals(request.getNamespace()));
+        }
+        if (request.getServiceName() != null) {
+            filter = filter.and(v -> v.id().serviceName().equals(request.getServiceName()));
+        }
+        final ServiceAccountInfo[] serviceAccountInfos = ServiceAccountService.getServiceAccounts()
+            .values()
+            .stream()
+            .filter(filter)
+            .map(v -> new ServiceAccountInfo(v.id().asPrincipal(), v.roleDescriptor()))
+            .toArray(ServiceAccountInfo[]::new);
+        listener.onResponse(new GetServiceAccountResponse(serviceAccountInfos));
     }
 }

x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/authc/service/ServiceAccountService.java

@@ -18,10 +18,10 @@ import org.elasticsearch.rest.RestStatus;
 import org.elasticsearch.xpack.core.security.action.service.CreateServiceAccountTokenRequest;
 import org.elasticsearch.xpack.core.security.action.service.CreateServiceAccountTokenResponse;
 import org.elasticsearch.xpack.core.security.action.service.DeleteServiceAccountTokenRequest;
+import org.elasticsearch.xpack.core.security.action.service.GetServiceAccountCredentialsNodesRequest;
 import org.elasticsearch.xpack.core.security.action.service.GetServiceAccountCredentialsRequest;
 import org.elasticsearch.xpack.core.security.action.service.GetServiceAccountCredentialsResponse;
 import org.elasticsearch.xpack.core.security.action.service.GetServiceAccountNodesCredentialsAction;
-import org.elasticsearch.xpack.core.security.action.service.GetServiceAccountCredentialsNodesRequest;
 import org.elasticsearch.xpack.core.security.action.service.TokenInfo;
 import org.elasticsearch.xpack.core.security.action.service.TokenInfo.TokenSource;
 import org.elasticsearch.xpack.core.security.authc.Authentication;
@@ -29,7 +29,6 @@ import org.elasticsearch.xpack.core.security.authc.service.ServiceAccountSetting
 import org.elasticsearch.xpack.core.security.authz.RoleDescriptor;
 import org.elasticsearch.xpack.core.security.user.User;
 import org.elasticsearch.xpack.security.authc.service.ServiceAccount.ServiceAccountId;
-import org.elasticsearch.xpack.security.authc.support.HttpTlsRuntimeCheck;
 
 import java.util.Collection;
 import java.util.List;
@@ -50,17 +49,14 @@ public class ServiceAccountService {
     private final Client client;
     private final IndexServiceAccountTokenStore indexServiceAccountTokenStore;
     private final CompositeServiceAccountTokenStore compositeServiceAccountTokenStore;
-    private final HttpTlsRuntimeCheck httpTlsRuntimeCheck;
 
     public ServiceAccountService(Client client,
                                  FileServiceAccountTokenStore fileServiceAccountTokenStore,
-                                 IndexServiceAccountTokenStore indexServiceAccountTokenStore,
-                                 HttpTlsRuntimeCheck httpTlsRuntimeCheck) {
+                                 IndexServiceAccountTokenStore indexServiceAccountTokenStore) {
         this.client = client;
         this.indexServiceAccountTokenStore = indexServiceAccountTokenStore;
         this.compositeServiceAccountTokenStore = new CompositeServiceAccountTokenStore(
             List.of(fileServiceAccountTokenStore, indexServiceAccountTokenStore), client.threadPool().getThreadContext());
-        this.httpTlsRuntimeCheck = httpTlsRuntimeCheck;
     }
 
     public static boolean isServiceAccountPrincipal(String principal) {
@@ -103,79 +99,76 @@ public class ServiceAccountService {
 
     public void authenticateToken(ServiceAccountToken serviceAccountToken, String nodeName, ActionListener<Authentication> listener) {
         logger.trace("attempt to authenticate service account token [{}]", serviceAccountToken.getQualifiedName());
-        httpTlsRuntimeCheck.checkTlsThenExecute(listener::onFailure, "service account authentication", () -> {
-            if (ElasticServiceAccounts.NAMESPACE.equals(serviceAccountToken.getAccountId().namespace()) == false) {
-                logger.debug("only [{}] service accounts are supported, but received [{}]",
-                    ElasticServiceAccounts.NAMESPACE, serviceAccountToken.getAccountId().asPrincipal());
-                listener.onFailure(createAuthenticationException(serviceAccountToken));
-                return;
-            }
 
-            final ServiceAccount account = ACCOUNTS.get(serviceAccountToken.getAccountId().asPrincipal());
-            if (account == null) {
-                logger.debug("the [{}] service account does not exist", serviceAccountToken.getAccountId().asPrincipal());
-                listener.onFailure(createAuthenticationException(serviceAccountToken));
-                return;
-            }
+        if (ElasticServiceAccounts.NAMESPACE.equals(serviceAccountToken.getAccountId().namespace()) == false) {
+            logger.debug(
+                "only [{}] service accounts are supported, but received [{}]",
+                ElasticServiceAccounts.NAMESPACE,
+                serviceAccountToken.getAccountId().asPrincipal()
+            );
+            listener.onFailure(createAuthenticationException(serviceAccountToken));
+            return;
+        }
 
-            if (serviceAccountToken.getSecret().length() < MIN_TOKEN_SECRET_LENGTH) {
-                logger.debug("failing authentication for service account token [{}],"
-                        + " the provided credential has length [{}]"
-                        + " but a token's secret value must be at least [{}] characters",
-                    serviceAccountToken.getQualifiedName(),
-                    serviceAccountToken.getSecret().length(),
-                    MIN_TOKEN_SECRET_LENGTH);
-                listener.onFailure(createAuthenticationException(serviceAccountToken));
-                return;
-            }
+        final ServiceAccount account = ACCOUNTS.get(serviceAccountToken.getAccountId().asPrincipal());
+        if (account == null) {
+            logger.debug("the [{}] service account does not exist", serviceAccountToken.getAccountId().asPrincipal());
+            listener.onFailure(createAuthenticationException(serviceAccountToken));
+            return;
+        }
 
-            compositeServiceAccountTokenStore.authenticate(serviceAccountToken, ActionListener.wrap(storeAuthenticationResult -> {
-                if (storeAuthenticationResult.isSuccess()) {
-                    listener.onResponse(
-                        createAuthentication(account, serviceAccountToken, storeAuthenticationResult.getTokenSource() , nodeName));
-                } else {
-                    final ElasticsearchSecurityException e = createAuthenticationException(serviceAccountToken);
-                    logger.debug(e.getMessage());
-                    listener.onFailure(e);
-                }
-            }, listener::onFailure));
-        });
+        if (serviceAccountToken.getSecret().length() < MIN_TOKEN_SECRET_LENGTH) {
+            logger.debug(
+                "failing authentication for service account token [{}],"
+                    + " the provided credential has length [{}]"
+                    + " but a token's secret value must be at least [{}] characters",
+                serviceAccountToken.getQualifiedName(),
+                serviceAccountToken.getSecret().length(),
+                MIN_TOKEN_SECRET_LENGTH
+            );
+            listener.onFailure(createAuthenticationException(serviceAccountToken));
+            return;
+        }
+
+        compositeServiceAccountTokenStore.authenticate(serviceAccountToken, ActionListener.wrap(storeAuthenticationResult -> {
+            if (storeAuthenticationResult.isSuccess()) {
+                listener.onResponse(
+                    createAuthentication(account, serviceAccountToken, storeAuthenticationResult.getTokenSource(), nodeName)
+                );
+            } else {
+                final ElasticsearchSecurityException e = createAuthenticationException(serviceAccountToken);
+                logger.debug(e.getMessage());
+                listener.onFailure(e);
+            }
+        }, listener::onFailure));
     }
 
     public void createIndexToken(Authentication authentication, CreateServiceAccountTokenRequest request,
                                  ActionListener<CreateServiceAccountTokenResponse> listener) {
-        httpTlsRuntimeCheck.checkTlsThenExecute(listener::onFailure,
-            "create index-backed service token",
-            () -> indexServiceAccountTokenStore.createToken(authentication, request, listener));
+        indexServiceAccountTokenStore.createToken(authentication, request, listener);
     }
 
     public void deleteIndexToken(DeleteServiceAccountTokenRequest request, ActionListener<Boolean> listener) {
-        httpTlsRuntimeCheck.checkTlsThenExecute(
-            listener::onFailure,
-            "delete index-backed service token",
-            () -> indexServiceAccountTokenStore.deleteToken(request, listener));
+       indexServiceAccountTokenStore.deleteToken(request, listener);
     }
 
     public void findTokensFor(GetServiceAccountCredentialsRequest request,
                               ActionListener<GetServiceAccountCredentialsResponse> listener) {
-        httpTlsRuntimeCheck.checkTlsThenExecute(listener::onFailure, "find service tokens", () -> {
-            final ServiceAccountId accountId = new ServiceAccountId(request.getNamespace(), request.getServiceName());
-            findIndexTokens(accountId, listener);
-        });
+        final ServiceAccountId accountId = new ServiceAccountId(request.getNamespace(), request.getServiceName());
+        findIndexTokens(accountId, listener);
     }
 
     public void getRoleDescriptor(Authentication authentication, ActionListener<RoleDescriptor> listener) {
         assert authentication.isServiceAccount() : "authentication is not for service account: " + authentication;
-        httpTlsRuntimeCheck.checkTlsThenExecute(listener::onFailure, "service account role descriptor resolving", () -> {
-            final String principal = authentication.getUser().principal();
-            final ServiceAccount account = ACCOUNTS.get(principal);
-            if (account == null) {
-                listener.onFailure(new ElasticsearchSecurityException(
-                    "cannot load role for service account [" + principal + "] - no such service account"));
-                return;
-            }
-            listener.onResponse(account.roleDescriptor());
-        });
+        final String principal = authentication.getUser().principal();
+        final ServiceAccount account = ACCOUNTS.get(principal);
+        if (account == null) {
+            listener.onFailure(
+                new ElasticsearchSecurityException("cannot load role for service account [" + principal + "] - no such service account")
+            );
+            return;
+        }
+        listener.onResponse(account.roleDescriptor());
     }
 
     private Authentication createAuthentication(ServiceAccount account, ServiceAccountToken token, TokenSource tokenSource,

x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/authc/support/HttpTlsRuntimeCheck.java

@@ -1,67 +0,0 @@
-/*
- * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
- * or more contributor license agreements. Licensed under the Elastic License
- * 2.0; you may not use this file except in compliance with the Elastic License
- * 2.0.
- */
-
-package org.elasticsearch.xpack.security.authc.support;
-
-import org.apache.logging.log4j.LogManager;
-import org.apache.logging.log4j.Logger;
-import org.apache.logging.log4j.message.ParameterizedMessage;
-import org.apache.lucene.util.SetOnce;
-import org.elasticsearch.ElasticsearchException;
-import org.elasticsearch.common.settings.Settings;
-import org.elasticsearch.discovery.DiscoveryModule;
-import org.elasticsearch.transport.Transport;
-import org.elasticsearch.xpack.core.XPackSettings;
-
-import java.util.Arrays;
-import java.util.concurrent.atomic.AtomicBoolean;
-import java.util.function.Consumer;
-
-public class HttpTlsRuntimeCheck {
-
-    private static final Logger logger = LogManager.getLogger(HttpTlsRuntimeCheck.class);
-
-    private final AtomicBoolean initialized = new AtomicBoolean(false);
-    private final Boolean httpTlsEnabled;
-    private final SetOnce<Transport> transportReference;
-    private final Boolean securityEnabled;
-    private final boolean singleNodeDiscovery;
-    private boolean enforce;
-
-    public HttpTlsRuntimeCheck(Settings settings, SetOnce<Transport> transportReference) {
-        this.transportReference = transportReference;
-        this.securityEnabled = XPackSettings.SECURITY_ENABLED.get(settings);
-        this.httpTlsEnabled = XPackSettings.HTTP_SSL_ENABLED.get(settings);
-        this.singleNodeDiscovery = "single-node".equals(DiscoveryModule.DISCOVERY_TYPE_SETTING.get(settings));
-    }
-
-    public void checkTlsThenExecute(Consumer<Exception> exceptionConsumer, String featureName, Runnable andThen) {
-        // If security is enabled, but TLS is not enabled for the HTTP interface
-        if (securityEnabled && false == httpTlsEnabled) {
-            if (false == initialized.get()) {
-                final Transport transport = transportReference.get();
-                if (transport == null) {
-                    exceptionConsumer.accept(new ElasticsearchException("transport cannot be null"));
-                    return;
-                }
-                final boolean boundToLocal = Arrays.stream(transport.boundAddress().boundAddresses())
-                    .allMatch(b -> b.address().getAddress().isLoopbackAddress())
-                    && transport.boundAddress().publishAddress().address().getAddress().isLoopbackAddress();
-                this.enforce = false == boundToLocal && false == singleNodeDiscovery;
-                initialized.set(true);
-            }
-            if (enforce) {
-                final ParameterizedMessage message = new ParameterizedMessage(
-                    "[{}] requires TLS for the HTTP interface", featureName);
-                logger.debug(message);
-                exceptionConsumer.accept(new ElasticsearchException(message.getFormattedMessage()));
-                return;
-            }
-        }
-        andThen.run();
-    }
-}

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/ApiKeySSLBootstrapCheckTests.java

@@ -1,33 +0,0 @@
-/*
- * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
- * or more contributor license agreements. Licensed under the Elastic License
- * 2.0; you may not use this file except in compliance with the Elastic License
- * 2.0.
- */
-
-package org.elasticsearch.xpack.security;
-
-import org.elasticsearch.common.settings.Settings;
-import org.elasticsearch.test.AbstractBootstrapCheckTestCase;
-import org.elasticsearch.xpack.core.XPackSettings;
-
-public class ApiKeySSLBootstrapCheckTests extends AbstractBootstrapCheckTestCase {
-
-    public void testApiKeySSLBootstrapCheck() {
-        Settings settings = Settings.EMPTY;
-
-        assertTrue(new ApiKeySSLBootstrapCheck().check(createTestContext(settings, null)).isSuccess());
-
-        settings = Settings.builder().put(XPackSettings.HTTP_SSL_ENABLED.getKey(), true).build();
-        assertTrue(new ApiKeySSLBootstrapCheck().check(createTestContext(settings, null)).isSuccess());
-
-        // XPackSettings.HTTP_SSL_ENABLED default false
-        settings = Settings.builder().put(XPackSettings.API_KEY_SERVICE_ENABLED_SETTING.getKey(), true).build();
-        assertTrue(new ApiKeySSLBootstrapCheck().check(createTestContext(settings, null)).isFailure());
-
-        settings = Settings.builder()
-                .put(XPackSettings.HTTP_SSL_ENABLED.getKey(), true)
-                .put(XPackSettings.API_KEY_SERVICE_ENABLED_SETTING.getKey(), true).build();
-        assertTrue(new ApiKeySSLBootstrapCheck().check(createTestContext(settings, null)).isSuccess());
-    }
-}

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/action/service/TransportGetServiceAccountActionTests.java

@@ -15,7 +15,6 @@ import org.elasticsearch.transport.TransportService;
 import org.elasticsearch.xpack.core.security.action.service.GetServiceAccountRequest;
 import org.elasticsearch.xpack.core.security.action.service.GetServiceAccountResponse;
 import org.elasticsearch.xpack.core.security.action.service.ServiceAccountInfo;
-import org.elasticsearch.xpack.security.authc.support.HttpTlsRuntimeCheck;
 import org.junit.Before;
 
 import java.util.Arrays;
@@ -24,26 +23,18 @@ import java.util.stream.Collectors;
 
 import static org.hamcrest.Matchers.containsInAnyOrder;
 import static org.hamcrest.Matchers.equalTo;
-import static org.mockito.Matchers.any;
-import static org.mockito.Mockito.doAnswer;
 import static org.mockito.Mockito.mock;
 
 public class TransportGetServiceAccountActionTests extends ESTestCase {
 
-    private HttpTlsRuntimeCheck httpTlsRuntimeCheck;
     private TransportGetServiceAccountAction transportGetServiceAccountAction;
 
     @Before
     public void init() {
-        httpTlsRuntimeCheck = mock(HttpTlsRuntimeCheck.class);
         transportGetServiceAccountAction = new TransportGetServiceAccountAction(
-            mock(TransportService.class), new ActionFilters(Collections.emptySet()), httpTlsRuntimeCheck);
-
-        doAnswer(invocationOnMock -> {
-            final Object[] arguments = invocationOnMock.getArguments();
-            ((Runnable) arguments[2]).run();
-            return null;
-        }).when(httpTlsRuntimeCheck).checkTlsThenExecute(any(), any(), any());
+            mock(TransportService.class),
+            new ActionFilters(Collections.emptySet())
+        );
     }
 
     public void testDoExecute() {

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/service/ServiceAccountServiceTests.java

@@ -10,8 +10,6 @@ package org.elasticsearch.xpack.security.authc.service;
 import org.apache.logging.log4j.Level;
 import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
-import org.apache.lucene.util.SetOnce;
-import org.elasticsearch.ElasticsearchException;
 import org.elasticsearch.ElasticsearchSecurityException;
 import org.elasticsearch.Version;
 import org.elasticsearch.action.ActionListener;
@@ -21,22 +19,19 @@ import org.elasticsearch.common.Strings;
 import org.elasticsearch.common.logging.Loggers;
 import org.elasticsearch.common.settings.SecureString;
 import org.elasticsearch.common.settings.Settings;
-import org.elasticsearch.common.transport.BoundTransportAddress;
-import org.elasticsearch.common.transport.TransportAddress;
 import org.elasticsearch.core.SuppressForbidden;
 import org.elasticsearch.test.ESTestCase;
 import org.elasticsearch.test.MockLogAppender;
 import org.elasticsearch.threadpool.TestThreadPool;
 import org.elasticsearch.threadpool.ThreadPool;
-import org.elasticsearch.transport.Transport;
 import org.elasticsearch.xpack.core.security.action.service.CreateServiceAccountTokenRequest;
 import org.elasticsearch.xpack.core.security.action.service.CreateServiceAccountTokenResponse;
 import org.elasticsearch.xpack.core.security.action.service.DeleteServiceAccountTokenRequest;
+import org.elasticsearch.xpack.core.security.action.service.GetServiceAccountCredentialsNodesRequest;
+import org.elasticsearch.xpack.core.security.action.service.GetServiceAccountCredentialsNodesResponse;
 import org.elasticsearch.xpack.core.security.action.service.GetServiceAccountCredentialsRequest;
 import org.elasticsearch.xpack.core.security.action.service.GetServiceAccountCredentialsResponse;
 import org.elasticsearch.xpack.core.security.action.service.GetServiceAccountNodesCredentialsAction;
-import org.elasticsearch.xpack.core.security.action.service.GetServiceAccountCredentialsNodesRequest;
-import org.elasticsearch.xpack.core.security.action.service.GetServiceAccountCredentialsNodesResponse;
 import org.elasticsearch.xpack.core.security.action.service.TokenInfo;
 import org.elasticsearch.xpack.core.security.authc.Authentication;
 import org.elasticsearch.xpack.core.security.authc.service.ServiceAccountSettings;
@@ -44,13 +39,11 @@ import org.elasticsearch.xpack.core.security.authz.RoleDescriptor;
 import org.elasticsearch.xpack.core.security.support.ValidationTests;
 import org.elasticsearch.xpack.core.security.user.User;
 import org.elasticsearch.xpack.security.authc.service.ServiceAccount.ServiceAccountId;
-import org.elasticsearch.xpack.security.authc.support.HttpTlsRuntimeCheck;
 import org.junit.After;
 import org.junit.Before;
 
 import java.io.ByteArrayOutputStream;
 import java.io.IOException;
-import java.net.InetAddress;
 import java.net.UnknownHostException;
 import java.nio.charset.StandardCharsets;
 import java.util.Arrays;
@@ -82,7 +75,6 @@ public class ServiceAccountServiceTests extends ESTestCase {
     private FileServiceAccountTokenStore fileServiceAccountTokenStore;
     private IndexServiceAccountTokenStore indexServiceAccountTokenStore;
     private ServiceAccountService serviceAccountService;
-    private Transport transport;
 
     @Before
     @SuppressForbidden(reason = "Allow accessing localhost")
@@ -94,25 +86,9 @@ public class ServiceAccountServiceTests extends ESTestCase {
         when(indexServiceAccountTokenStore.getTokenSource()).thenReturn(TokenInfo.TokenSource.INDEX);
         final Settings.Builder builder = Settings.builder()
             .put("xpack.security.enabled", true);
-        transport = mock(Transport.class);
-        final TransportAddress transportAddress;
-        if (randomBoolean()) {
-            transportAddress = new TransportAddress(TransportAddress.META_ADDRESS, 9300);
-        } else {
-            transportAddress = new TransportAddress(InetAddress.getLocalHost(), 9300);
-        }
-        if (randomBoolean()) {
-            builder.put("xpack.security.http.ssl.enabled", true);
-        } else {
-            builder.put("discovery.type", "single-node");
-        }
-        when(transport.boundAddress()).thenReturn(
-            new BoundTransportAddress(new TransportAddress[] { transportAddress }, transportAddress));
         client = mock(Client.class);
         when(client.threadPool()).thenReturn(threadPool);
-        serviceAccountService = new ServiceAccountService(client,
-            fileServiceAccountTokenStore, indexServiceAccountTokenStore,
-            new HttpTlsRuntimeCheck(builder.build(), new SetOnce<>(transport)));
+        serviceAccountService = new ServiceAccountService(client, fileServiceAccountTokenStore, indexServiceAccountTokenStore);
     }
 
     @After
@@ -536,53 +512,6 @@ public class ServiceAccountServiceTests extends ESTestCase {
         assertThat(response.getIndexTokenInfos(), equalTo(indexTokenInfos));
     }
 
-    public void testTlsRequired() {
-        final Settings settings = Settings.builder()
-            .put("xpack.security.http.ssl.enabled", false)
-            .build();
-        final TransportAddress transportAddress = new TransportAddress(TransportAddress.META_ADDRESS, 9300);
-        when(transport.boundAddress()).thenReturn(
-            new BoundTransportAddress(new TransportAddress[] { transportAddress }, transportAddress));
-
-        final ServiceAccountService service = new ServiceAccountService(client,
-            fileServiceAccountTokenStore,indexServiceAccountTokenStore,
-            new HttpTlsRuntimeCheck(settings, new SetOnce<>(transport)));
-
-        final PlainActionFuture<Authentication> future1 = new PlainActionFuture<>();
-        service.authenticateToken(mock(ServiceAccountToken.class), randomAlphaOfLengthBetween(3, 8), future1);
-        final ElasticsearchException e1 = expectThrows(ElasticsearchException.class, future1::actionGet);
-        assertThat(e1.getMessage(), containsString("[service account authentication] requires TLS for the HTTP interface"));
-
-        final PlainActionFuture<RoleDescriptor> future2 = new PlainActionFuture<>();
-        final TokenInfo.TokenSource tokenSource = randomFrom(TokenInfo.TokenSource.values());
-        final Authentication authentication = new Authentication(mock(User.class),
-            new Authentication.RealmRef(
-                ServiceAccountSettings.REALM_NAME, ServiceAccountSettings.REALM_TYPE,
-                randomAlphaOfLengthBetween(3, 8)),
-            null,
-            Version.CURRENT,
-            Authentication.AuthenticationType.TOKEN,
-            Map.of("_token_name", randomAlphaOfLengthBetween(3, 8), "_token_source", tokenSource.name().toLowerCase(Locale.ROOT)));
-        service.getRoleDescriptor(authentication, future2);
-        final ElasticsearchException e2 = expectThrows(ElasticsearchException.class, future2::actionGet);
-        assertThat(e2.getMessage(), containsString("[service account role descriptor resolving] requires TLS for the HTTP interface"));
-
-        final PlainActionFuture<CreateServiceAccountTokenResponse> future3 = new PlainActionFuture<>();
-        service.createIndexToken(authentication, mock(CreateServiceAccountTokenRequest.class), future3);
-        final ElasticsearchException e3 = expectThrows(ElasticsearchException.class, future3::actionGet);
-        assertThat(e3.getMessage(), containsString("[create index-backed service token] requires TLS for the HTTP interface"));
-
-        final PlainActionFuture<Boolean> future4 = new PlainActionFuture<>();
-        service.deleteIndexToken(mock(DeleteServiceAccountTokenRequest.class), future4);
-        final ElasticsearchException e4 = expectThrows(ElasticsearchException.class, future4::actionGet);
-        assertThat(e4.getMessage(), containsString("[delete index-backed service token] requires TLS for the HTTP interface"));
-
-        final PlainActionFuture<GetServiceAccountCredentialsResponse> future5 = new PlainActionFuture<>();
-        service.findTokensFor(mock(GetServiceAccountCredentialsRequest.class), future5);
-        final ElasticsearchException e5 = expectThrows(ElasticsearchException.class, future5::actionGet);
-        assertThat(e5.getMessage(), containsString("[find service tokens] requires TLS for the HTTP interface"));
-    }
-
     private SecureString createBearerString(List<byte[]> bytesList) throws IOException {
         try (ByteArrayOutputStream out = new ByteArrayOutputStream()) {
             for (byte[] bytes : bytesList) {