Rewrite the scripting security docs (#23930)

They needed to be updated now that Painless is the default and
the non-sandboxed scripting languages are going away or gone.

I dropped the entire section about customizing the classloader
whitelists. In master this barely does anything (exposes more
things to expressions).

docs/build.gradle

@@ -87,7 +87,6 @@ buildRestTests.expectedUnconvertedCandidates = [
   'reference/mapping/types/nested.asciidoc',
   'reference/mapping/types/object.asciidoc',
   'reference/mapping/types/percolator.asciidoc',
-  'reference/modules/scripting/security.asciidoc',
   'reference/modules/cross-cluster-search.asciidoc', // this is hard to test since we need 2 clusters -- maybe we can trick it into referencing itself...
   'reference/search/field-stats.asciidoc',
   'reference/search/profile.asciidoc',

docs/plugins/authors.asciidoc

@@ -76,6 +76,7 @@ Read more in {ref}/integration-tests.html#changing-node-configuration[Changing N
 
 
 [float]
+[[plugin-authors-jsm]]
 === Java Security permissions
 
 Some plugins may need additional security permissions. A plugin can include
@@ -111,4 +112,3 @@ AccessController.doPrivileged(
 
 See http://www.oracle.com/technetwork/java/seccodeguide-139067.html[Secure Coding Guidelines for Java SE]
 for more information.
-

docs/reference/modules/scripting/security.asciidoc

@@ -1,73 +1,115 @@
 [[modules-scripting-security]]
 === Scripting and security
 
-You should never run Elasticsearch as the `root` user, as this would allow a
-script to access or do *anything* on your server, without limitations.
+While Elasticsearch contributors make every effort to prevent scripts from
+running amok, security is something best done in
+https://en.wikipedia.org/wiki/Defense_in_depth_(computing)[layers] because
+all software has bugs and it is important to minimize the risk of failure in
+any security layer. Find below rules of thumb for how to keep Elasticsearch
+from being a vulnerability.
 
-You should not expose Elasticsearch directly to users, but instead have a
-proxy application inbetween. If you *do* intend to expose Elasticsearch
-directly to your users, then you have to decide whether you trust them enough
-to run scripts on your box or not, and apply the appropriate safety measures.
-
-[[enable-dynamic-scripting]]
 [float]
-=== Enabling dynamic scripting
+=== Do not run as root
+First and foremost, never run Elasticsearch as the `root` user as this would
+allow any successful effort to circumvent the other security layers to do
+*anything* on your server. Elasticsearch will refuse to start if it detects
+that it is running as `root` but this is so important that it is worth double
+and triple checking.
 
-The `script.*` settings allow for <<security-script-fine,fine-grained>>
-control of which script languages (e.g `painless`) are allowed to
-run in which context ( e.g. `search`, `aggs`, `update`), and where the script
-source is allowed to come from (i.e. `inline`, `stored`, `file`).
+[float]
+=== Do not expose Elasticsearch directly to users
+Do not expose Elasticsearch directly to users, instead have an application
+make requests on behalf of users. If this is not possible, have an application
+to sanitize requests from users. If *that* is not possible then have some
+mechanism to track which users did what. Understand that it is quite possible
+to write a <<search, `_search`>> that overwhelms Elasticsearch and brings down
+the cluster. All such searches should be considered bugs and the Elasticsearch
+contributors make an effort to prevent this but they are still possible.
 
-For instance, the following setting enables `stored` `update` scripts for
-`painless`:
+[float]
+=== Do not expose Elasticsearch directly to the Internet
+Do not expose Elasticsearch to the Internet, instead have an application
+make requests on behalf of the Internet. Do not entertain the thought of having
+an application "sanitize" requests to Elasticsearch. Understand that it is
+possible for a sufficiently determined malicious user to write searches that
+overwhelm the Elasticsearch cluster and bring it down. For example:
+
+Good:
+* Users type text into a search box and the text is sent directly to a
+<<query-dsl-match-query>>, <<query-dsl-match-query-phrase>>,
+<<query-dsl-simple-query-string-query>>, or any of the <<search-suggesters>>.
+* Running a script with any of the above queries that was written as part of
+the application development process.
+* Running a script with `params` provided by users.
+* User actions makes documents with a fixed structure.
+
+Bad:
+* Users can write arbitrary scripts, queries, `_search` requests.
+* User actions make documents with structure defined by users.
 
-[source,yaml]
-----------------
-script.engine.painless.inline.update: true
-----------------
+[float]
+[[modules-scripting-security-do-no-weaken]]
+=== Do not weaken script security settings
+By default Elasticsearch will run inline, stored, and filesystem scripts for
+sandboxed languages, namely the scripting language Painless, the template
+language Mustache, and the expression language Expressions. These *ought* to be
+safe to expose to trusted users and to your application servers because they
+have strong security sandboxes. By default Elasticsearch will only run
+filesystem scripts for non-sandboxed languages and enabling them is a poor
+choice because:
+1. This drops a layer of security, leaving only Elasticsearch's builtin
+<<modules-scripting-other-layers, security layers>>.
+2. Non-sandboxed scripts have unchecked access to Elasticsearch's internals and
+can cause all kinds of trouble if misused.
 
-Less fine-grained settings exist which allow you to enable or disable scripts
-for all sources, all languages, or all contexts.  The following settings
-enable `inline` and `stored` scripts for all languages in all contexts:
 
-[source,yaml]
------------------------------------
-script.inline: true
-script.stored: true
------------------------------------
-
-WARNING:  The above settings mean that anybody who can send requests to your
-Elasticsearch instance can run whatever scripts they choose! This is a
-security risk and may well lead to your Elasticsearch cluster being
-compromised.
+[float]
+[[modules-scripting-other-layers]]
+=== Other security layers
+In addition to user privileges and script sandboxing Elasticsearch uses the
+http://www.oracle.com/technetwork/java/seccodeguide-139067.html[Java Security Manager]
+and native security tools as additional layers of security.
+
+As part of its startup sequence Elasticsearch enables the Java Security Manager
+which limits the actions that can be taken by portions of the code. Painless
+uses this to limit the actions that generated Painless scripts can take,
+preventing them from being able to do things like write files and listen to
+sockets.
+
+Elasticsearch uses
+https://en.wikipedia.org/wiki/Seccomp[seccomp] in Linux,
+https://www.chromium.org/developers/design-documents/sandbox/osx-sandboxing-design[Seatbelt]
+in macOS, and
+https://msdn.microsoft.com/en-us/library/windows/desktop/ms684147[ActiveProcessLimit]
+on Windows to prevent Elasticsearch from forking or executing other processes.
+
+Below this we describe the security settings for scripts and how you can
+change from the defaults described above. You should be very, very careful
+when allowing more than the defaults. Any extra permissions weakens the total
+security of the Elasticsearch deployment.
 
 [[security-script-source]]
 [float]
 === Script source settings
 
-Scripts may be enabled or disabled depending on their source: `inline`,
-`stored` in the cluster state, or from a `file` on each node in the cluster.
-Each of these settings takes one of these values:
-
-
-[horizontal]
-`false`::   Scripting is disabled.
-`true`::    Scripting is enabled.
-
-The default values are the following:
+Which scripts Elasticsearch will execute where is controlled by settings
+starting with `scripts.`. The simplest settings allow scripts to be enabled
+or disabled based on where they are stored. For example:
 
 [source,yaml]
 -----------------------------------
-script.inline: false
-script.stored: false
-script.file:   true
+script.inline: false  <1>
+script.stored: false  <2>
+script.file:   true   <3>
 -----------------------------------
+<1> Refuse to run scripts provided inline in the API.
+<2> Refuse to run scripts stored using the API.
+<3> Run scripts found on the filesystem in `/etc/elasticsearch/scripts`
+(rpm or deb) or `config/scripts` (zip or tar).
 
-NOTE: Global scripting settings affect the `mustache` scripting language.
-<<search-template,Search templates>> internally use the `mustache` language,
-and will still be enabled by default as the `mustache` engine is sandboxed,
-but they will be enabled/disabled according to fine-grained settings
-specified in `elasticsearch.yml`.
+NOTE: These settings override the defaults mentioned
+<<modules-scripting-security-do-no-weaken, above>>. Recreating the defaults
+requires more fine grained settings described <<security-script-fine, below>>.
 
 [[security-script-context]]
 [float]
@@ -102,15 +144,13 @@ script.plugin: false
 === Fine-grained script settings
 
 First, the high-level script settings described above are applied in order
-(context settings have precedence over source settings).  Then,  fine-grained
+(context settings have precedence over source settings). Then fine-grained
 settings which include the script language take precedence over any high-level
-settings.
-
-Fine-grained settings have the form:
+settings. They have two forms:
 
 [source,yaml]
 ------------------------
-script.engine.{lang}.{source}.{context}: true|false
+script.engine.{lang}.{inline|file|stored}.{context}: true|false
 ------------------------
 
 And
@@ -132,124 +172,9 @@ script.engine.painless.inline:          true <2>
 script.engine.painless.stored.search:   true <3>
 script.engine.painless.stored.aggs:     true <3>
 
-script.engine.mustache.stored.search: true <4>
+script.engine.mustache.stored.search:   true <4>
 -----------------------------------
 <1> Disable all scripting from any source.
-<2> Allow inline Groovy scripts for all operations
-<3> Allow stored Groovy scripts to be used for search and aggregations.
+<2> Allow inline Painless scripts for all operations.
+<3> Allow stored Painless scripts to be used for search and aggregations.
 <4> Allow stored Mustache templates to be used for search.
-
-[[java-security-manager]]
-[float]
-=== Java Security Manager
-
-Elasticsearch runs with the https://docs.oracle.com/javase/tutorial/essential/environment/security.html[Java Security Manager]
-enabled by default.  The security policy in Elasticsearch locks down the
-permissions granted to each class to the bare minimum required to operate.
-The benefit of doing this is that it severely limits the attack vectors
-available to a hacker.
-
-Restricting permissions is particularly important with scripting languages
-like Groovy which is designed to do anything that can be done
-in Java itself, including writing to the file system, opening sockets to
-remote servers, etc.
-
-[float]
-=== Script Classloader Whitelist
-
-Scripting languages are only allowed to load classes which appear in a
-hardcoded whitelist that can be found in
-https://github.com/elastic/elasticsearch/blob/{branch}/core/src/main/java/org/elasticsearch/script/ClassPermission.java[`org.elasticsearch.script.ClassPermission`].
-
-
-In a script, attempting to load a class that does not appear in the whitelist
-_may_ result in a `ClassNotFoundException`, for instance this script:
-
-[source,js]
-------------------------------
-GET _search
-{
-  "script_fields": {
-    "the_hour": {
-      "script": "use(java.math.BigInteger); new BigInteger(1)"
-    }
-  }
-}
-------------------------------
-
-will return the following exception:
-
-[source,js]
-------------------------------
-{
-  "reason": {
-    "type": "script_exception",
-    "reason": "failed to run inline script [use(java.math.BigInteger); new BigInteger(1)] using lang [painless]",
-    "caused_by": {
-      "type": "no_class_def_found_error",
-      "reason": "java/math/BigInteger",
-      "caused_by": {
-        "type": "class_not_found_exception",
-        "reason": "java.math.BigInteger"
-      }
-    }
-  }
-}
-------------------------------
-
-[float]
-== Dealing with Java Security Manager issues
-
-If you encounter issues with the Java Security Manager, you have two options
-for resolving these issues:
-
-[float]
-=== Fix the security problem
-
-The safest and most secure long term solution is to change the code causing
-the security issue.  We recognise that this may take time to do correctly and
-so we provide the following two alternatives.
-
-[float]
-=== Customising the classloader whitelist
-
-The classloader whitelist can be customised by tweaking the local Java
-Security Policy either:
-
-* system wide: `$JAVA_HOME/lib/security/java.policy`,
-* for just the `elasticsearch` user: `/home/elasticsearch/.java.policy`
-* by adding a system property to the <<jvm-options,jvm.options>> configuration: `-Djava.security.policy=someURL`, or
-* via the `ES_JAVA_OPTS` environment variable with `-Djava.security.policy=someURL`:
-+
-[source,js]
----------------------------------
-export ES_JAVA_OPTS="${ES_JAVA_OPTS} -Djava.security.policy=file:///path/to/my.policy`
-./bin/elasticsearch
----------------------------------
-
-Permissions may be granted at the class, package, or global level.  For instance:
-
-[source,js]
-----------------------------------
-grant {
-    permission org.elasticsearch.script.ClassPermission "java.util.Base64"; // allow class
-    permission org.elasticsearch.script.ClassPermission "java.util.*"; // allow package
-    permission org.elasticsearch.script.ClassPermission "*"; // allow all (disables filtering basically)
-};
-----------------------------------
-
-[TIP]
-======================================
-
-Before adding classes to the whitelist, consider the security impact that it
-will have on Elasticsearch. Do you really need an extra class or can your code
-be rewritten in a more secure way?
-
-It is quite possible that we have not whitelisted a generically useful and
-safe class. If you have a class that you think should be whitelisted by
-default, please open an issue on GitHub and we will consider the impact of
-doing so.
-
-======================================
-
-See http://docs.oracle.com/javase/7/docs/technotes/guides/security/PolicyFiles.html for more information.

docs/reference/search/search-template.asciidoc

@@ -28,8 +28,8 @@ documentation of the mustache project].
 
 NOTE: The mustache language is implemented in elasticsearch as a sandboxed
 scripting language, hence it obeys settings that may be used to enable or
-disable scripts per language, source and operation as described in
-<<enable-dynamic-scripting, scripting docs>>
+disable scripts per language, source and operation as described in the
+<<security-script-source, scripting docs>>
 
 [float]
 ==== More template examples