Documentation for operator privileges (#68964)

Add documentation for operator privilegs. The docs cover features delivered by phase 1 (#65256) and 2 (#66684).

Co-authored-by: Tim Vernum <tim@adjective.org>
Co-authored-by: lcawl <lcawley@elastic.co>
Co-authored-by: Adam Locke <adam.locke@elastic.co>

x-pack/docs/en/security/index.asciidoc

@@ -17,6 +17,7 @@ IP filtering, and auditing.
 * <<enable-audit-logging>>
 * <<encrypting-communications>>
 * <<ccs-clients-integrations>>
+* <<operator-privileges>>
 * <<security-getting-started>>
 * <<encrypting-internode-communications>>
 * <<security-troubleshooting>>
@@ -40,6 +41,8 @@ include::using-ip-filtering.asciidoc[]
 
 include::ccs-clients-integrations/index.asciidoc[]
 
+include::operator-privileges/index.asciidoc[]
+
 include::get-started-security.asciidoc[]
 
 include::securing-communications/tutorial-tls-intro.asciidoc[]

x-pack/docs/en/security/operator-privileges/configure-operator-privileges.asciidoc

@@ -0,0 +1,85 @@
+[role="xpack"]
+[testenv="enterprise"]
+[[configure-operator-privileges]]
+=== Configure operator privileges
+
+include::operator-privileges-designed-for-note.asciidoc[]
+
+Before you can use operator privileges, you must
+<<enable-operator-privileges, enable the feature>> on all nodes in the cluster
+and <<designate-operator-users,designate operator users>>.
+
+[[enable-operator-privileges]]
+==== Enable operator privileges
+
+In order to use the operator privileges feature, it must be enabled explicitly
+on each node in the cluster. Add the following setting in each
+`elasticsearch.yml` file:
+
+[source,yaml]
+----------------------------
+xpack.security.operator_privileges.enabled: true
+----------------------------
+
+If the node is already running before you make this change, you must restart
+the node for the feature to take effect.
+
+WARNING: The feature needs to be either enabled or disabled consistently across
+all nodes in a cluster. Otherwise, you can get inconsistent behaviour depending
+on which node first receives a request and which node executes it.
+
+When operator privileges are enabled on a cluster,
+<<operator-only-functionality,specific functionalities>> are restricted and can
+be executed only by users who have been explicitly designated as operator users.
+If a regular user attempts to execute these functionalities (even if they have
+the `superuser` role), a security exception occurs.
+
+[[designate-operator-users]]
+==== Designate operator users
+
+Operator users are just normal {es} users with special rights to perform
+operator-only functionalities. They are specified in an `operator_users.yml`
+file, which is located in the config directory (as defined by the `ES_PATH_CONF`
+environment variable). Similar to
+<<file-realm-configuration,other security config files>>, the
+`operator_users.yml` file is local to a node and does not apply globally to the
+cluster. This means, in most cases, the same file should be distributed or
+copied to all nodes in a cluster.
+
+The `operator_users.yml` file defines a set of criteria that an authenticating
+user must match to be considered as an operator user. The following snippet
+shows an example of such a file:
+
+[source,yaml]
+-----------------------------------
+operator: <1>
+  - usernames: ["system_agent_1","system_agent_2"] <2>
+    realm_type: "file" <3>
+    auth_type: "realm" <4>
+-----------------------------------
+<1> A fixed value of `operator` signals the beginning of the definition.
+<2> A list of user names allowed for operator users. This field is mandatory.
+<3> The type of the authenticating realm allowed for operator users. The default
+and only acceptable value is <<file-realm,`file`>>.
+<4> The authentication type allowed for operator users. The default and only
+acceptable value is `realm`.
+
+You must specify at least the `usernames` field. If no other fields are
+specified, their default values are used. All fields must be matched for a user
+to be qualified as an operator user. You can also specify multiple groups of
+criteria. This is currently not very useful since this feature does not yet
+support other realms or authentication types.
+
+There are also two implicit rules that affect which users are operator users:
+
+1. If the authenticating user <<run-as-privilege,runs as>> another user, neither
+of them are considered to be operator users.
+2. All <<internal-users>> are implicitly operator users.
+
+IMPORTANT: After a user is designated as an operator user, they are still
+subject to regular <<authorization,RBAC user authorization>> checks. That is to
+say, in addition to specifying that a user is an operator user, you must also
+grant them the necessary {es} roles to perform their tasks. Consequently, it is
+entirely possible that an operator user can encounter an "access denied" error
+and fail to perform certain actions due to RBAC check failures. In short, an
+operator user is *not* automatically a `superuser`.

x-pack/docs/en/security/operator-privileges/index.asciidoc

@@ -0,0 +1,28 @@
+[role="xpack"]
+[testenv="enterprise"]
+[[operator-privileges]]
+== Operator privileges
+
+include::operator-privileges-designed-for-note.asciidoc[]
+
+With a typical {es} deployment, people who administer the cluster also operate
+the cluster at the infrastructure level. User authorization based on
+<<authorization,role-based access control (RBAC)>> is effective and reliable for
+this environment. However, in more managed environments, such as
+{ess-trial}[{ess}], there is a distinction between the operator of the cluster
+infrastructure and the administrator of the cluster. 
+
+Operator privileges limit some functionality to operator users only. Operator
+users are just regular Elasticsearch users with access to specific
+<<operator-only-functionality,operator-only functionality>>. These
+privileges are not available to cluster administrators, even if they log in as
+a highly privileged user such as the `elastic` user or another user with the
+superuser role. By limiting system access, operator privileges enhance the
+Elasticsearch security model while safeguarding user capabilities.
+
+
+include::configure-operator-privileges.asciidoc[]
+
+include::operator-only-functionality.asciidoc[]
+
+include::operator-only-snapshot-and-restore.asciidoc[]

x-pack/docs/en/security/operator-privileges/operator-only-functionality.asciidoc

@@ -0,0 +1,40 @@
+[role="xpack"]
+[testenv="enterprise"]
+[[operator-only-functionality]]
+=== Operator-only functionality
+
+include::operator-privileges-designed-for-note.asciidoc[]
+
+Operator privileges provide protection for APIs and dynamic cluster settings.
+Any API or cluster setting that is protected by operator privileges is known as 
+_operator-only functionality_. When the operator privileges feature is enabled, 
+operator-only APIs can be executed only by operator users. Likewise,
+operator-only settings can be updated only by operator users. The list of
+operator-only APIs and dynamic cluster settings are pre-determined in the
+codebase. The list may evolve in future releases but it is otherwise fixed in a
+given {es} version.
+
+[[operator-only-apis]]
+==== Operator-only APIs
+
+* <<voting-config-exclusions>>
+* <<delete-license>>
+* <<update-license>>
+* <<autoscaling-put-autoscaling-policy>>
+* <<autoscaling-delete-autoscaling-policy>>
+* <<repo-analysis-api>>
+
+[[operator-only-dynamic-cluster-settings]]
+==== Operator-only dynamic cluster settings
+
+* All <<ip-filtering,IP filtering>> settings
+* The following the dynamic <<ml-settings,machine learning settings>>:
+  - `xpack.ml.node_concurrent_job_allocations`
+  - `xpack.ml.max_machine_memory_percent`
+  - `xpack.ml.use_auto_machine_memory_percent`
+  - `xpack.ml.max_lazy_ml_nodes`
+  - `xpack.ml.process_connect_timeout`
+  - `xpack.ml.nightly_maintenance_requests_per_second`
+  - `xpack.ml.max_ml_node_size`
+  - `xpack.ml.enable_config_migration`
+  - `xpack.ml.persist_results_max_retries`

x-pack/docs/en/security/operator-privileges/operator-only-snapshot-and-restore.asciidoc

@@ -0,0 +1,41 @@
+[role="xpack"]
+[testenv="enterprise"]
+[[operator-only-snapshot-and-restore]]
+=== Operator privileges for snapshot and restore
+
+include::operator-privileges-designed-for-note.asciidoc[]
+
+Invoking <<operator-only-apis,operator-only APIs>> or updating
+<<operator-only-dynamic-cluster-settings,operator-only dynamic cluster settings>>
+typically results in changes in the cluster state. The cluster state can be
+included in a cluster <<snapshot-restore,snapshot>>. Snapshots are a great way
+to preserve the data of a cluster, which can later be restored to bootstrap a
+new cluster, perform migration, or disaster recovery, for example. In a
+traditional self-managed environment, the intention is for the restore process
+to copy the entire cluster state over when requested. However, in a more
+managed environment, such as {ess-trial}[{ess}], data that is associated with
+<<operator-only-functionality,operator-only functionality>> is explicitly
+managed by the infrastructure code.
+
+Restoring snapshot data associated with
+operator-only functionality could be problematic
+because:
+
+1. A snapshot could contain incorrect values for operator-only functionalities.
+For example, the snapshot could have been taken in a different cluster where 
+requirements are different or the operator privileges feature is not enabled. 
+Restoring data associated with operator-only functionality breaks the guarantee
+of operator privileges.
+2. Even when the infrastructure code can correct the values immediately after
+a restore, there will always be a short period of time when the cluster could be
+in an inconsistent state.
+3. The infrastructure code prefers to configure operator-only functionality from
+a single place, that is to say, through API calls.
+
+Therefore,
+<<configure-operator-privileges,*when the operator privileges feature is enabled*>>,
+snapshot data that is associated with any operator-only functionality is *not* 
+restored.
+
+NOTE: That information is still included when taking a snapshot so that all data
+is always preserved.

x-pack/docs/en/security/operator-privileges/operator-privileges-designed-for-note.asciidoc

@@ -0,0 +1,4 @@
+[NOTE]
+The operator privileges feature is designed for indirect use by {ess-trial}[{ess}],
+{ece-ref}[{ece}], and {eck-ref}[Elastic Cloud on Kubernetes]. Direct use is not
+supported.