Support Profile Activate with JWTs with client authn (#105439)

Adds support for JWTs with client authentication
to the activate user profile API.

Closes #105342

docs/changelog/105439.yaml

@@ -0,0 +1,6 @@
+pr: 105439
+summary: Support Profile Activate with JWTs with client authn
+area: Authentication
+type: enhancement
+issues:
+ - 105342

docs/reference/rest-api/security/activate-user-profile.asciidoc

@@ -28,6 +28,9 @@ Creates or updates a user profile on behalf of another user.
 The activate user profile API creates or updates a profile document for end
 users with information that is extracted from the user's authentication object,
 including `username`, `full_name`, `roles`, and the authentication realm.
+For example, in the JWT `access_token` case, the profile user's `username` is
+extracted from the JWT token claim pointed to by the `claims.principal`
+setting of the JWT realm that authenticated the token.
 
 When updating a profile document, the API enables the document if it was
 disabled. Any updates do not change existing content for either the `labels` or
@@ -46,8 +49,11 @@ is intended for.
 
 `access_token`::
 (Required*, string)
-The user's access token. If you specify the `access_token` grant type, this
-parameter is required. It is not valid with other grant types.
+The user's <<security-api-get-token, {es} access token>>, or JWT. Both <<jwt-realm-oauth2, access>> and
+<<jwt-realm-oidc, id>> JWT token types are supported, and they depend on the underlying JWT realm configuration.
+If you specify the `access_token` grant type, this parameter is required. It is not valid with other grant types.
+
+include::client-authentication.asciidoc[]
 
 `grant_type`::
 (Required, string)
@@ -57,24 +63,22 @@ The type of grant.
 [%collapsible%open]
 ====
 `access_token`::
-(Required*, string)
-In this type of grant, you must supply an access token that was created by the
-{es} token service. For more information, see
-<<security-api-get-token>> and <<token-service-settings>>.
+In this type of grant, you must supply either an access token, that was created by the
+{es} token service (see <<security-api-get-token>> and <<encrypt-http-communication>>),
+or a <<jwt-auth-realm, JWT>> (either a JWT `access_token` or a JWT `id_token`).
 
 `password`::
-(Required*, string)
 In this type of grant, you must supply the `username` and `password` for the
 user that you want to create the API key for.
 ====
 
 `password`::
-(Optional*, string)
+(Required*, string)
 The user's password. If you specify the `password` grant type, this parameter is
 required. It is not valid with other grant types.
 
 `username`::
-(Optional*, string)
+(Required*, string)
 The username that identifies the user. If you specify the `password` grant type,
 this parameter is required. It is not valid with other grant types.
 

docs/reference/rest-api/security/client-authentication.asciidoc

@@ -0,0 +1,16 @@
+`client_authentication`::
+(Optional, object) When using the `access_token` grant type, and when supplying a
+JWT, this specifies the client authentication for <<jwt-auth-realm, JWTs>> that
+need it (i.e. what's normally specified by the `ES-Client-Authentication` request header).
+
+`scheme`:::
+(Required, string) The scheme (case-sensitive) as it's supplied in the
+`ES-Client-Authentication` request header. Currently, the only supported
+value is <<jwt-auth-shared-secret-scheme-example, `SharedSecret`>>.
+
+`value`:::
+(Required, string) The value that follows the scheme for the client credentials
+as it's supplied in the `ES-Client-Authentication` request header. For example,
+if the request header would be `ES-Client-Authentication: SharedSecret myShar3dS3cret`
+if the client were to authenticate directly with a JWT, then `value` here should
+be `myShar3dS3cret`.

docs/reference/rest-api/security/grant-api-keys.asciidoc

@@ -89,29 +89,13 @@ It supports nested data structure.
 Within the `metadata` object, keys beginning with `_` are reserved for
 system usage.
 
-`client_authentication`::
-(Optional, object) When using the `access_token` grant type, and when supplying a
-JWT, this specifies the client authentication for <<jwt-auth-realm, JWTs>> that
-need it (i.e. what's normally specified by the `ES-Client-Authentication` request header).
-
-`scheme`:::
-(Required, string) The scheme (case-sensitive) as it's supplied in the
-`ES-Client-Authentication` request header. Currently, the only supported
-value is <<jwt-auth-shared-secret-scheme-example, `SharedSecret`>>.
-
-`value`:::
-(Required, string) The value that follows the scheme for the client credentials
-as it's supplied in the `ES-Client-Authentication` request header. For example,
-if the request header would be `ES-Client-Authentication: SharedSecret myShar3dS3cret`
-if the client were to authenticate directly with a JWT, then `value` here should
-be `myShar3dS3cret`.
+include::client-authentication.asciidoc[]
 
 `grant_type`::
 (Required, string)
 The type of grant. Supported grant types are: `access_token`,`password`.
 
 `access_token`:::
-(Required*, string)
 In this type of grant, you must supply either an access token, that was created by the
 {es} token service (see <<security-api-get-token>> and <<encrypt-http-communication>>),
 or a <<jwt-auth-realm, JWT>> (either a JWT `access_token` or a JWT `id_token`).
@@ -121,12 +105,12 @@ In this type of grant, you must supply the user ID and password for which you
 want to create the API key.
 
 `password`::
-(Optional*, string)
+(Required*, string)
 The user's password. If you specify the `password` grant type, this parameter is
 required. It is not valid with other grant types.
 
 `username`::
-(Optional*, string)
+(Required*, string)
 The user name that identifies the user. If you specify the `password` grant type,
 this parameter is required. It is not valid with other grant types.
 
@@ -134,6 +118,8 @@ this parameter is required. It is not valid with other grant types.
 (Optional, string)
 The name of the user to be <<run-as-privilege,impersonated>>.
 
+*Indicates that the setting is required in some, but not all situations.
+
 [[security-api-grant-api-key-example]]
 ==== {api-examples-title}
 

x-pack/plugin/security/src/internalClusterTest/java/org/elasticsearch/xpack/security/authc/jwt/JwtRealmSingleNodeTests.java

@@ -41,6 +41,12 @@ import org.elasticsearch.xpack.core.security.action.Grant;
 import org.elasticsearch.xpack.core.security.action.apikey.CreateApiKeyResponse;
 import org.elasticsearch.xpack.core.security.action.apikey.GrantApiKeyAction;
 import org.elasticsearch.xpack.core.security.action.apikey.GrantApiKeyRequest;
+import org.elasticsearch.xpack.core.security.action.profile.ActivateProfileAction;
+import org.elasticsearch.xpack.core.security.action.profile.ActivateProfileRequest;
+import org.elasticsearch.xpack.core.security.action.profile.ActivateProfileResponse;
+import org.elasticsearch.xpack.core.security.action.profile.GetProfilesAction;
+import org.elasticsearch.xpack.core.security.action.profile.GetProfilesRequest;
+import org.elasticsearch.xpack.core.security.action.profile.GetProfilesResponse;
 import org.elasticsearch.xpack.core.security.action.user.AuthenticateAction;
 import org.elasticsearch.xpack.core.security.action.user.AuthenticateRequest;
 import org.elasticsearch.xpack.core.security.action.user.AuthenticateResponse;
@@ -237,6 +243,84 @@ public class JwtRealmSingleNodeTests extends SecuritySingleNodeTestCase {
         }
     }
 
+    public void testActivateProfileForJWT() throws Exception {
+        final JWTClaimsSet.Builder jwtClaims = new JWTClaimsSet.Builder();
+        final String principal;
+        final String sharedSecret;
+        final String realmName;
+        // id_token or access_token
+        if (randomBoolean()) {
+            principal = "me";
+            // JWT "id_token" valid for jwt0
+            jwtClaims.audience("es-01")
+                .issuer("my-issuer-01")
+                .subject(principal)
+                .claim("groups", "admin")
+                .issueTime(Date.from(Instant.now()))
+                .expirationTime(Date.from(Instant.now().plusSeconds(600)))
+                .build();
+            sharedSecret = jwt0SharedSecret;
+            realmName = "jwt0";
+        } else {
+            principal = "me@example.com";
+            // JWT "access_token" valid for jwt2
+            jwtClaims.audience("es-03")
+                .issuer("my-issuer-03")
+                .subject("user-03")
+                .claim("groups", "admin")
+                .claim("email", principal)
+                .issueTime(Date.from(Instant.now()))
+                .expirationTime(Date.from(Instant.now().plusSeconds(300)));
+            sharedSecret = jwt2SharedSecret;
+            realmName = "jwt2";
+        }
+        {
+            // JWT is valid but the client authentication is NOT
+            ActivateProfileRequest activateProfileRequest = getActivateProfileForJWT(
+                getSignedJWT(jwtClaims.build()),
+                randomFrom("WRONG", null)
+            );
+            ElasticsearchSecurityException e = expectThrows(
+                ElasticsearchSecurityException.class,
+                () -> client().execute(ActivateProfileAction.INSTANCE, activateProfileRequest).actionGet()
+            );
+            assertThat(e.getMessage(), containsString("unable to authenticate user"));
+        }
+        {
+            // both JWT and client authentication are valid
+            ActivateProfileRequest activateProfileRequest = getActivateProfileForJWT(getSignedJWT(jwtClaims.build()), sharedSecret);
+            ActivateProfileResponse activateProfileResponse = client().execute(ActivateProfileAction.INSTANCE, activateProfileRequest)
+                .actionGet();
+            assertThat(activateProfileResponse.getProfile(), notNullValue());
+            assertThat(activateProfileResponse.getProfile().uid(), notNullValue());
+            assertThat(activateProfileResponse.getProfile().user().username(), is(principal));
+            assertThat(activateProfileResponse.getProfile().user().realmName(), is(realmName));
+            // test to get the profile by uid
+            GetProfilesRequest getProfilesRequest = new GetProfilesRequest(List.of(activateProfileResponse.getProfile().uid()), Set.of());
+            GetProfilesResponse getProfilesResponse = client().execute(GetProfilesAction.INSTANCE, getProfilesRequest).actionGet();
+            assertThat(getProfilesResponse.getProfiles().size(), is(1));
+            assertThat(getProfilesResponse.getProfiles().get(0).uid(), is(activateProfileResponse.getProfile().uid()));
+            assertThat(getProfilesResponse.getProfiles().get(0).enabled(), is(true));
+            assertThat(getProfilesResponse.getProfiles().get(0).user().username(), is(principal));
+            assertThat(getProfilesResponse.getProfiles().get(0).user().realmName(), is(realmName));
+        }
+        {
+            // client authentication is valid but the JWT is not
+            final SignedJWT wrongJWT;
+            if (randomBoolean()) {
+                wrongJWT = getSignedJWT(jwtClaims.build(), ("wrong key that's longer than 256 bits").getBytes(StandardCharsets.UTF_8));
+            } else {
+                wrongJWT = getSignedJWT(jwtClaims.audience("wrong audience claim value").build());
+            }
+            ActivateProfileRequest activateProfileRequest = getActivateProfileForJWT(wrongJWT, sharedSecret);
+            ElasticsearchSecurityException e = expectThrows(
+                ElasticsearchSecurityException.class,
+                () -> client().execute(ActivateProfileAction.INSTANCE, activateProfileRequest).actionGet()
+            );
+            assertThat(e.getMessage(), containsString("unable to authenticate user"));
+        }
+    }
+
     @SuppressWarnings("unchecked")
     public void testInvalidJWTDoesNotFallbackToAnonymousAccess() throws Exception {
         // anonymous access works when no valid Bearer
@@ -696,4 +780,15 @@ public class JwtRealmSingleNodeTests extends SecuritySingleNodeTestCase {
         grantApiKeyRequest.getApiKeyRequest().setName(randomAlphaOfLength(8));
         return grantApiKeyRequest;
     }
+
+    private static ActivateProfileRequest getActivateProfileForJWT(SignedJWT signedJWT, String sharedSecret) {
+        ActivateProfileRequest activateProfileRequest = new ActivateProfileRequest();
+        activateProfileRequest.getGrant().setType("access_token");
+        activateProfileRequest.getGrant().setAccessToken(new SecureString(signedJWT.serialize().toCharArray()));
+        if (sharedSecret != null) {
+            activateProfileRequest.getGrant()
+                .setClientAuthentication(new Grant.ClientAuthentication("SharedSecret", new SecureString(sharedSecret.toCharArray())));
+        }
+        return activateProfileRequest;
+    }
 }

x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/rest/action/SecurityBaseRestHandler.java

@@ -7,14 +7,21 @@
 package org.elasticsearch.xpack.security.rest.action;
 
 import org.elasticsearch.client.internal.node.NodeClient;
+import org.elasticsearch.common.settings.SecureString;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.license.XPackLicenseState;
 import org.elasticsearch.rest.BaseRestHandler;
 import org.elasticsearch.rest.RestRequest;
 import org.elasticsearch.rest.RestResponse;
+import org.elasticsearch.xcontent.ConstructingObjectParser;
+import org.elasticsearch.xcontent.ObjectParser;
+import org.elasticsearch.xcontent.ParseField;
+import org.elasticsearch.xcontent.XContentParser;
 import org.elasticsearch.xpack.core.XPackSettings;
+import org.elasticsearch.xpack.core.security.action.Grant;
 
 import java.io.IOException;
+import java.util.Arrays;
 
 /**
  * Base class for security rest handlers. This handler takes care of ensuring that the license
@@ -22,6 +29,25 @@ import java.io.IOException;
  */
 public abstract class SecurityBaseRestHandler extends BaseRestHandler {
 
+    protected static final ConstructingObjectParser<Grant.ClientAuthentication, Void> CLIENT_AUTHENTICATION_PARSER =
+        new ConstructingObjectParser<>("client_authentication", a -> new Grant.ClientAuthentication((String) a[0], (SecureString) a[1]));
+
+    static {
+        CLIENT_AUTHENTICATION_PARSER.declareString(ConstructingObjectParser.constructorArg(), new ParseField("scheme"));
+        CLIENT_AUTHENTICATION_PARSER.declareField(
+            ConstructingObjectParser.constructorArg(),
+            SecurityBaseRestHandler::getSecureString,
+            new ParseField("value"),
+            ObjectParser.ValueType.STRING
+        );
+    }
+
+    protected static SecureString getSecureString(XContentParser parser) throws IOException {
+        return new SecureString(
+            Arrays.copyOfRange(parser.textCharacters(), parser.textOffset(), parser.textOffset() + parser.textLength())
+        );
+    }
+
     protected final Settings settings;
     protected final XPackLicenseState licenseState;
 

x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/rest/action/apikey/RestGrantApiKeyAction.java

@@ -11,7 +11,6 @@ import org.elasticsearch.ElasticsearchSecurityException;
 import org.elasticsearch.ExceptionsHelper;
 import org.elasticsearch.action.support.WriteRequest;
 import org.elasticsearch.client.internal.node.NodeClient;
-import org.elasticsearch.common.settings.SecureString;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.license.XPackLicenseState;
 import org.elasticsearch.rest.RestRequest;
@@ -20,19 +19,17 @@ import org.elasticsearch.rest.RestStatus;
 import org.elasticsearch.rest.Scope;
 import org.elasticsearch.rest.ServerlessScope;
 import org.elasticsearch.rest.action.RestToXContentListener;
-import org.elasticsearch.xcontent.ConstructingObjectParser;
 import org.elasticsearch.xcontent.ObjectParser;
 import org.elasticsearch.xcontent.ParseField;
 import org.elasticsearch.xcontent.XContentParser;
-import org.elasticsearch.xpack.core.security.action.Grant;
 import org.elasticsearch.xpack.core.security.action.apikey.CreateApiKeyRequestBuilder;
 import org.elasticsearch.xpack.core.security.action.apikey.CreateApiKeyResponse;
 import org.elasticsearch.xpack.core.security.action.apikey.GrantApiKeyAction;
 import org.elasticsearch.xpack.core.security.action.apikey.GrantApiKeyRequest;
 import org.elasticsearch.xpack.security.authc.ApiKeyService;
+import org.elasticsearch.xpack.security.rest.action.SecurityBaseRestHandler;
 
 import java.io.IOException;
-import java.util.Arrays;
 import java.util.List;
 import java.util.Set;
 
@@ -46,31 +43,19 @@ import static org.elasticsearch.rest.RestRequest.Method.PUT;
 @ServerlessScope(Scope.INTERNAL)
 public final class RestGrantApiKeyAction extends ApiKeyBaseRestHandler implements RestRequestFilter {
 
-    private static final ConstructingObjectParser<Grant.ClientAuthentication, Void> CLIENT_AUTHENTICATION_PARSER =
-        new ConstructingObjectParser<>("client_authentication", a -> new Grant.ClientAuthentication((String) a[0], (SecureString) a[1]));
-    static {
-        CLIENT_AUTHENTICATION_PARSER.declareString(ConstructingObjectParser.constructorArg(), new ParseField("scheme"));
-        CLIENT_AUTHENTICATION_PARSER.declareField(
-            ConstructingObjectParser.constructorArg(),
-            RestGrantApiKeyAction::getSecureString,
-            new ParseField("value"),
-            ObjectParser.ValueType.STRING
-        );
-    }
-
     static final ObjectParser<GrantApiKeyRequest, Void> PARSER = new ObjectParser<>("grant_api_key_request", GrantApiKeyRequest::new);
     static {
         PARSER.declareString((req, str) -> req.getGrant().setType(str), new ParseField("grant_type"));
         PARSER.declareString((req, str) -> req.getGrant().setUsername(str), new ParseField("username"));
         PARSER.declareField(
             (req, secStr) -> req.getGrant().setPassword(secStr),
-            RestGrantApiKeyAction::getSecureString,
+            SecurityBaseRestHandler::getSecureString,
             new ParseField("password"),
             ObjectParser.ValueType.STRING
         );
         PARSER.declareField(
             (req, secStr) -> req.getGrant().setAccessToken(secStr),
-            RestGrantApiKeyAction::getSecureString,
+            SecurityBaseRestHandler::getSecureString,
             new ParseField("access_token"),
             ObjectParser.ValueType.STRING
         );
@@ -87,12 +72,6 @@ public final class RestGrantApiKeyAction extends ApiKeyBaseRestHandler implement
         );
     }
 
-    private static SecureString getSecureString(XContentParser parser) throws IOException {
-        return new SecureString(
-            Arrays.copyOfRange(parser.textCharacters(), parser.textOffset(), parser.textOffset() + parser.textLength())
-        );
-    }
-
     public RestGrantApiKeyAction(Settings settings, XPackLicenseState licenseState) {
         super(settings, licenseState);
     }
@@ -138,10 +117,8 @@ public final class RestGrantApiKeyAction extends ApiKeyBaseRestHandler implement
         }
     }
 
-    private static final Set<String> FILTERED_FIELDS = Set.of("password", "access_token", "client_authentication.value");
-
     @Override
     public Set<String> getFilteredFields() {
-        return FILTERED_FIELDS;
+        return Set.of("password", "access_token", "client_authentication.value");
     }
 }

x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/rest/action/profile/RestActivateProfileAction.java

@@ -8,7 +8,6 @@
 package org.elasticsearch.xpack.security.rest.action.profile;
 
 import org.elasticsearch.client.internal.node.NodeClient;
-import org.elasticsearch.common.settings.SecureString;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.license.XPackLicenseState;
 import org.elasticsearch.rest.RestRequest;
@@ -24,7 +23,6 @@ import org.elasticsearch.xpack.core.security.action.profile.ActivateProfileReque
 import org.elasticsearch.xpack.security.rest.action.SecurityBaseRestHandler;
 
 import java.io.IOException;
-import java.util.Arrays;
 import java.util.List;
 import java.util.Set;
 
@@ -42,16 +40,21 @@ public class RestActivateProfileAction extends SecurityBaseRestHandler implement
         PARSER.declareString((req, str) -> req.getGrant().setUsername(str), new ParseField("username"));
         PARSER.declareField(
             (req, secStr) -> req.getGrant().setPassword(secStr),
-            RestActivateProfileAction::getSecureString,
+            SecurityBaseRestHandler::getSecureString,
             new ParseField("password"),
             ObjectParser.ValueType.STRING
         );
         PARSER.declareField(
             (req, secStr) -> req.getGrant().setAccessToken(secStr),
-            RestActivateProfileAction::getSecureString,
+            SecurityBaseRestHandler::getSecureString,
             new ParseField("access_token"),
             ObjectParser.ValueType.STRING
         );
+        PARSER.declareObject(
+            (req, clientAuthentication) -> req.getGrant().setClientAuthentication(clientAuthentication),
+            CLIENT_AUTHENTICATION_PARSER,
+            new ParseField("client_authentication")
+        );
     }
 
     public RestActivateProfileAction(Settings settings, XPackLicenseState licenseState) {
@@ -68,23 +71,21 @@ public class RestActivateProfileAction extends SecurityBaseRestHandler implement
         return "xpack_security_activate_profile";
     }
 
+    // package-private for tests
+    static ActivateProfileRequest fromXContent(XContentParser parser) throws IOException {
+        return PARSER.parse(parser, null);
+    }
+
     @Override
     protected RestChannelConsumer innerPrepareRequest(RestRequest request, NodeClient client) throws IOException {
         try (XContentParser parser = request.contentParser()) {
-            final ActivateProfileRequest activateProfileRequest = PARSER.parse(parser, null);
+            final ActivateProfileRequest activateProfileRequest = fromXContent(parser);
             return channel -> client.execute(ActivateProfileAction.INSTANCE, activateProfileRequest, new RestToXContentListener<>(channel));
         }
     }
 
-    // TODO: extract to standalone helper method
-    private static SecureString getSecureString(XContentParser parser) throws IOException {
-        return new SecureString(
-            Arrays.copyOfRange(parser.textCharacters(), parser.textOffset(), parser.textOffset() + parser.textLength())
-        );
-    }
-
     @Override
     public Set<String> getFilteredFields() {
-        return Set.of("password", "access_token");
+        return Set.of("password", "access_token", "client_authentication.value");
     }
 }

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/rest/action/profile/RestActivateProfileActionTests.java

@@ -0,0 +1,54 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+package org.elasticsearch.xpack.security.rest.action.profile;
+
+import org.elasticsearch.common.settings.SecureString;
+import org.elasticsearch.test.ESTestCase;
+import org.elasticsearch.xcontent.XContentFactory;
+import org.elasticsearch.xcontent.XContentParser;
+import org.elasticsearch.xpack.core.security.action.profile.ActivateProfileRequest;
+
+import static org.hamcrest.Matchers.is;
+
+public class RestActivateProfileActionTests extends ESTestCase {
+
+    public void testParseXContentForGrantApiKeyRequest() throws Exception {
+        final String grantType = randomAlphaOfLength(8);
+        final String username = randomAlphaOfLength(8);
+        final String password = randomAlphaOfLength(8);
+        final String accessToken = randomAlphaOfLength(8);
+        final String clientAuthenticationScheme = randomAlphaOfLength(8);
+        final String clientAuthenticationValue = randomAlphaOfLength(8);
+        try (
+            XContentParser content = createParser(
+                XContentFactory.jsonBuilder()
+                    .startObject()
+                    .field("grant_type", grantType)
+                    .field("username", username)
+                    .field("password", password)
+                    .field("access_token", accessToken)
+                    .startObject("client_authentication")
+                    .field("scheme", clientAuthenticationScheme)
+                    .field("value", clientAuthenticationValue)
+                    .endObject()
+                    .endObject()
+            )
+        ) {
+            ActivateProfileRequest activateProfileRequest = RestActivateProfileAction.fromXContent(content);
+            assertThat(activateProfileRequest.getGrant().getType(), is(grantType));
+            assertThat(activateProfileRequest.getGrant().getUsername(), is(username));
+            assertThat(activateProfileRequest.getGrant().getPassword(), is(new SecureString(password.toCharArray())));
+            assertThat(activateProfileRequest.getGrant().getAccessToken(), is(new SecureString(accessToken.toCharArray())));
+            assertThat(activateProfileRequest.getGrant().getClientAuthentication().scheme(), is(clientAuthenticationScheme));
+            assertThat(
+                activateProfileRequest.getGrant().getClientAuthentication().value(),
+                is(new SecureString(clientAuthenticationValue.toCharArray()))
+            );
+        }
+    }
+}