Switch to Groovy as the default scripting language

This is a breaking change to move from MVEL -> Groovy

docs/reference/modules/advanced-scripting.asciidoc

@@ -37,12 +37,12 @@ depending on the shard the current document resides in.
 
 `_index.numDocs()`::
 
-    Number of documents in shard. 
-    
+    Number of documents in shard.
+
 `_index.maxDoc()`::
 
     Maximal document number in shard.
-    
+
 `_index.numDeletedDocs()`::
 
     Number of deleted documents in shard.
@@ -62,7 +62,7 @@ Field statistics can be accessed with a subscript operator like this:
 `_index['FIELD'].sumttf()`::
 
     Sum of `ttf` over all terms that appear in field `FIELD` in all documents.
-    
+
 `_index['FIELD'].sumdf()`::
 
     The sum of `df` s over all terms that appear in field `FIELD` in all
@@ -77,7 +77,7 @@ The number of terms in a field cannot be accessed using the `_index` variable. S
 === Term statistics:
 
 Term statistics for a field can be accessed with a subscript operator like
-this: `_index['FIELD']['TERM']`. This will never return null, even if term or field does not exist. 
+this: `_index['FIELD']['TERM']`. This will never return null, even if term or field does not exist.
 If you do not need the term frequency, call `_index['FIELD'].get('TERM', 0)`
 to avoid uneccesary initialization of the frequencies. The flag will have only
 affect is your set the `index_options` to `docs` (see <<mapping-core-types, mapping documentation>>).
@@ -162,11 +162,11 @@ Positions can be accessed with an iterator that returns an object
 
 Example: sums up all payloads for the term `foo`.
 
-[source,mvel]
+[source,groovy]
 ---------------------------------------------------------
 termInfo = _index['my_field'].get('foo',_PAYLOADS);
 score = 0;
-for (pos : termInfo) {
+for (pos in termInfo) {
     score = score + pos.payloadAsInt(0);
 }
 return score;
@@ -181,4 +181,3 @@ The `_index` variable can only be used to gather statistics for single terms. If
 https://lucene.apache.org/core/4_0_0/core/org/apache/lucene/index/Fields.html[Fields]
 instance. This object can then be used as described in https://lucene.apache.org/core/4_0_0/core/org/apache/lucene/index/Fields.html[lucene doc] to iterate over fields and then for each field iterate over each term in the field.
 The method will return null if the term vectors were not stored.
-

docs/reference/modules/scripting.asciidoc

@@ -6,28 +6,29 @@ expressions. For example, scripts can be used to return "script fields"
 as part of a search request, or can be used to evaluate a custom score
 for a query and so on.
 
-The scripting module uses by default http://mvel.codehaus.org/[mvel] as
-the scripting language with some extensions. mvel is used since it is
-extremely fast and very simple to use, and in most cases, simple
-expressions are needed (for example, mathematical equations).
+deprecated[1.3.0,Groovy has replaced Mvel as the default scripting language]
+
+The scripting module uses by default http://groovy.codehaus.org/[groovy]
+(previously http://mvel.codehaus.org/[mvel]) as the scripting language with some
+extensions. Groovy is used since it is extremely fast and very simple to use.
 
 Additional `lang` plugins are provided to allow to execute scripts in
 different languages. Currently supported plugins are `lang-javascript`
-for JavaScript, `lang-groovy` for Groovy, and `lang-python` for Python.
+for JavaScript, `lang-mvel` for Mvel, and `lang-python` for Python.
 All places where a `script` parameter can be used, a `lang` parameter
 (on the same level) can be provided to define the language of the
-script. The `lang` options are `mvel`, `js`, `groovy`, `python`, and
+script. The `lang` options are `groovy`, `js`, `mvel`, `python`, and
 `native`.
 
-added[1.2.0, Dynamic scripting is disabled by default since version 1.2.0]
+added[1.2.0, Dynamic scripting is disabled for non-sandboxed languages by default since version 1.2.0]
 
-To increase security, Elasticsearch does not allow you to specify scripts with a
-request. Instead, scripts must be placed in the `scripts` directory inside the
-configuration directory (the directory where elasticsearch.yml is). Scripts
-placed into this directory will automatically be picked up and be available to
-be used. Once a script has been placed in this directory, it can be referenced
-by name. For example, a script called `calculate-score.mvel` can be referenced
-in a request like this:
+To increase security, Elasticsearch does not allow you to specify scripts for
+non-sandboxed languages with a request. Instead, scripts must be placed in the
+`scripts` directory inside the configuration directory (the directory where
+elasticsearch.yml is). Scripts placed into this directory will automatically be
+picked up and be available to be used. Once a script has been placed in this
+directory, it can be referenced by name. For example, a script called
+`calculate-score.groovy` can be referenced in a request like this:
 
 [source,sh]
 --------------------------------------------------
@@ -36,13 +37,13 @@ config
 ├── elasticsearch.yml
 ├── logging.yml
 └── scripts
-    └── calculate-score.mvel
+    └── calculate-score.groovy
 --------------------------------------------------
 
 [source,sh]
 --------------------------------------------------
-$ cat config/scripts/calculate-score.mvel
-Math.log(_score * 2) + my_modifier
+$ cat config/scripts/calculate-score.groovy
+log(_score * 2) + my_modifier
 --------------------------------------------------
 
 [source,js]
@@ -75,21 +76,14 @@ exists under, and the file name without the lang extension. For example,
 a script placed under `config/scripts/group1/group2/test.py` will be
 named `group1_group2_test`.
 
-[float]
-=== Default Scripting Language
-
-The default scripting language (assuming no `lang` parameter is
-provided) is `mvel`. In order to change it set the `script.default_lang`
-to the appropriate language.
-
 [float]
 === Enabling dynamic scripting
 
-We recommend running Elasticsearch behind an application or proxy,
-which protects Elasticsearch from the outside world. If users are
-allowed to run dynamic scripts (even in a search request), then they
-have the same access to your box as the user that Elasticsearch is
-running as. For this reason dynamic scripting is disabled by default.
+We recommend running Elasticsearch behind an application or proxy, which
+protects Elasticsearch from the outside world. If users are allowed to run
+dynamic scripts (even in a search request), then they have the same access to
+your box as the user that Elasticsearch is running as. For this reason dynamic
+scripting is allowed only for sandboxed languages by default.
 
 First, you should not run Elasticsearch as the `root` user, as this would allow
 a script to access or do *anything* on your server, without limitations. Second,
@@ -109,6 +103,54 @@ _native_ Java scripts registered through plugins, it also allows users to run
 arbitrary scripts via the API. Instead of sending the name of the file as the
 script, the body of the script can be sent instead.
 
+There are three possible configuration values for the `script.disable_dynamic`
+setting, the default value is `sandbox`:
+
+[cols="<,<",options="header",]
+|=======================================================================
+|Value |Description
+| `true` |all dynamic scripting is disabled, scripts must be placed in the `config/scripts` directory.
+| `false` |all dynamic scripting is enabled, scripts may be sent as strings in requests.
+| `sandbox` |scripts may be sent as strings for languages that are sandboxed.
+|=======================================================================
+
+[float]
+=== Default Scripting Language
+
+The default scripting language (assuming no `lang` parameter is provided) is
+`groovy`. In order to change it, set the `script.default_lang` to the
+appropriate language.
+
+[float]
+=== Groovy Sandboxing
+
+Elasticsearch sandboxes Groovy scripts that are compiled and executed in order
+to ensure they don't perform unwanted actions. There are a number of options
+that can be used for configuring this sandbox:
+
+`script.groovy.sandbox.receiver_whitelist`::
+
+    Comma-separated list of string classes for objects that may have methods
+    invoked.
+
+`script.groovy.sandbox.package_whitelist`::
+
+    Comma-separated list of packages under which new objects may be constructed.
+
+`script.groovy.sandbox.class_whitelist`::
+
+    Comma-separated list of classes that are allowed to be constructed.
+
+`script.groovy.sandbox.method_blacklist`::
+
+    Comma-separated list of methods that are never allowed to be invoked,
+    regardless of target object.
+
+`script.groovy.sandbox.enabled`::
+
+    Flag to disable the sandbox (defaults to `true` meaning the sandbox is
+    enabled).
+
 [float]
 === Automatic Script Reloading
 
@@ -122,7 +164,7 @@ to `false`.
 [float]
 === Native (Java) Scripts
 
-Even though `mvel` is pretty fast, this allows to register native Java based
+Even though `groovy` is pretty fast, this allows to register native Java based
 scripts for faster execution.
 
 In order to allow for scripts, the `NativeScriptFactory` needs to be
@@ -267,7 +309,7 @@ loaded for other purposes.
 
 
 [float]
-=== mvel Built In Functions
+=== Groovy Built In Functions
 
 There are several built in functions that can be used within scripts.
 They include:
@@ -275,8 +317,6 @@ They include:
 [cols="<,<",options="header",]
 |=======================================================================
 |Function |Description
-|`time()` |The current time in milliseconds.
-
 |`sin(a)` |Returns the trigonometric sine of an angle.
 
 |`cos(a)` |Returns the trigonometric cosine of an angle.
@@ -350,6 +390,16 @@ power of the second argument.
 or underflow.
 |=======================================================================
 
+[float]
+=== Floating point numbers in Groovy
+
+When using floating-point literals in Groovy scripts, Groovy will automatically
+use BigDecimal instead of a floating point equivalent to support the
+'least-surprising' approach to literal math operations. To use a floating-point
+number instead, use the specific suffix on the number (ie, instead of 1.2, use
+1.2f). See the http://groovy.codehaus.org/Groovy+Math[Groovy Math] page for more
+information.
+
 [float]
 === Arithmetic precision in MVEL
 

rest-api-spec/api/update.json

@@ -34,7 +34,7 @@
         },
         "lang": {
           "type": "string",
-          "description": "The script language (default: mvel)"
+          "description": "The script language (default: groovy)"
         },
         "parent": {
           "type": "string",

rest-api-spec/test/update/15_script.yaml

@@ -17,7 +17,7 @@
           id:     1
           script: "1"
           body:
-            lang:   mvel
+            lang:   groovy
             script: "ctx._source.foo = bar"
             params: { bar: 'xxx' }
 
@@ -40,7 +40,7 @@
           index:  test_1
           type:   test
           id:     1
-          lang:   mvel
+          lang:   groovy
           script: "ctx._source.foo = 'yyy'"
 
   - match: { _index:   test_1 }

src/main/java/org/elasticsearch/action/update/UpdateRequestBuilder.java

@@ -92,7 +92,7 @@ public class UpdateRequestBuilder extends InstanceShardOperationRequestBuilder<U
     /**
      * The language of the script to execute.
      * Valid options are: mvel, js, groovy, python, and native (Java)<br>
-     * Default: mvel
+     * Default: groovy
      * <p>
      * Ref: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-scripting.html
      */

src/main/java/org/elasticsearch/script/ScriptService.java

@@ -110,7 +110,7 @@ public class ScriptService extends AbstractComponent {
         TimeValue cacheExpire = componentSettings.getAsTime("cache.expire", null);
         logger.debug("using script cache with max_size [{}], expire [{}]", cacheMaxSize, cacheExpire);
 
-        this.defaultLang = settings.get(DEFAULT_SCRIPTING_LANGUAGE_SETTING, "mvel");
+        this.defaultLang = settings.get(DEFAULT_SCRIPTING_LANGUAGE_SETTING, "groovy");
         this.dynamicScriptingDisabled = DynamicScriptDisabling.parse(settings.get(DISABLE_DYNAMIC_SCRIPTING_SETTING, DISABLE_DYNAMIC_SCRIPTING_DEFAULT));
 
         CacheBuilder cacheBuilder = CacheBuilder.newBuilder();

src/test/java/org/elasticsearch/script/IndexLookupTests.java

@@ -205,7 +205,7 @@ public class IndexLookupTests extends ElasticsearchIntegrationTest {
         initTestData();
 
         String script = "term = _index['float_payload_field'].get('b'," + includeAllFlag
-                + "); payloadSum=0; for (pos : term) {payloadSum = pos.payloadAsInt(0)}; payloadSum";
+                + "); payloadSum=0; for (pos in term) {payloadSum = pos.payloadAsInt(0)}; payloadSum";
 
         // non existing field: sum should be 0
         HashMap<String, Object> zeroArray = new HashMap<>();
@@ -215,7 +215,7 @@ public class IndexLookupTests extends ElasticsearchIntegrationTest {
         checkValueInEachDoc(script, zeroArray, 3);
 
         script = "term = _index['int_payload_field'].get('b'," + includeAllFlag
-                + "); payloadSum=0; for (pos : term) {payloadSum = payloadSum + pos.payloadAsInt(0)}; payloadSum";
+                + "); payloadSum=0; for (pos in term) {payloadSum = payloadSum + pos.payloadAsInt(0)}; payloadSum";
 
         // existing field: sums should be as here:
         zeroArray.put("1", 5);
@@ -263,26 +263,26 @@ public class IndexLookupTests extends ElasticsearchIntegrationTest {
 
     private String createPositionsArrayScriptGetInfoObjectTwice(String term, String flags, String what) {
         String script = "term = _index['int_payload_field'].get('" + term + "'," + flags
-                + "); array=[]; for (pos : term) {array.add(pos." + what + ")}; _index['int_payload_field'].get('" + term + "',"
-                + flags + "); array=[]; for (pos : term) {array.add(pos." + what + ")}";
+                + "); array=[]; for (pos in term) {array.add(pos." + what + ")}; _index['int_payload_field'].get('" + term + "',"
+                + flags + "); array=[]; for (pos in term) {array.add(pos." + what + ")}";
         return script;
     }
 
     private String createPositionsArrayScriptIterateTwice(String term, String flags, String what) {
         String script = "term = _index['int_payload_field'].get('" + term + "'," + flags
-                + "); array=[]; for (pos : term) {array.add(pos." + what + ")}; array=[]; for (pos : term) {array.add(pos." + what
+                + "); array=[]; for (pos in term) {array.add(pos." + what + ")}; array=[]; for (pos in term) {array.add(pos." + what
                 + ")}; array";
         return script;
     }
 
     private String createPositionsArrayScript(String field, String term, String flags, String what) {
         String script = "term = _index['" + field + "'].get('" + term + "'," + flags
-                + "); array=[]; for (pos : term) {array.add(pos." + what + ")}; array";
+                + "); array=[]; for (pos in term) {array.add(pos." + what + ")}; array";
         return script;
     }
 
     private String createPositionsArrayScriptDefaultGet(String field, String term, String what) {
-        String script = "term = _index['" + field + "']['" + term + "']; array=[]; for (pos : term) {array.add(pos." + what
+        String script = "term = _index['" + field + "']['" + term + "']; array=[]; for (pos in term) {array.add(pos." + what
                 + ")}; array";
         return script;
     }

src/test/java/org/elasticsearch/search/aggregations/bucket/TopHitsTests.java

@@ -269,7 +269,7 @@ public class TopHitsTests extends ElasticsearchIntegrationTest {
                                         topHits("hits").setSize(1)
                                 )
                                 .subAggregation(
-                                        max("max_score").script("_doc.score")
+                                        max("max_score").script("_doc.score()")
                                 )
                 )
                 .get();

src/test/java/org/elasticsearch/search/sort/SimpleSortTests.java

@@ -687,7 +687,7 @@ public class SimpleSortTests extends ElasticsearchIntegrationTest {
         // test the long values
         SearchResponse searchResponse = client().prepareSearch()
                 .setQuery(matchAllQuery())
-                .addScriptField("min", "retval = Long.MAX_VALUE; for (v : doc['lvalue'].values){ retval = min(v, retval) }; retval")
+                .addScriptField("min", "retval = Long.MAX_VALUE; for (v in doc['lvalue'].values){ retval = min(v, retval) }; retval")
                 .addSort("ord", SortOrder.ASC).setSize(10)
                 .execute().actionGet();
 
@@ -700,7 +700,7 @@ public class SimpleSortTests extends ElasticsearchIntegrationTest {
         // test the double values
         searchResponse = client().prepareSearch()
                 .setQuery(matchAllQuery())
-                .addScriptField("min", "retval = Double.MAX_VALUE; for (v : doc['dvalue'].values){ retval = min(v, retval) }; retval")
+                .addScriptField("min", "retval = Double.MAX_VALUE; for (v in doc['dvalue'].values){ retval = min(v, retval) }; retval")
                 .addSort("ord", SortOrder.ASC).setSize(10)
                 .execute().actionGet();
 
@@ -714,7 +714,7 @@ public class SimpleSortTests extends ElasticsearchIntegrationTest {
         // test the string values
         searchResponse = client().prepareSearch()
                 .setQuery(matchAllQuery())
-                .addScriptField("min", "retval = Integer.MAX_VALUE; for (v : doc['svalue'].values){ retval = min(Integer.parseInt(v), retval) }; retval")
+                .addScriptField("min", "retval = Integer.MAX_VALUE; for (v in doc['svalue'].values){ retval = min(Integer.parseInt(v), retval) }; retval")
                 .addSort("ord", SortOrder.ASC).setSize(10)
                 .execute().actionGet();
 
@@ -728,7 +728,7 @@ public class SimpleSortTests extends ElasticsearchIntegrationTest {
         // test the geopoint values
         searchResponse = client().prepareSearch()
                 .setQuery(matchAllQuery())
-                .addScriptField("min", "retval = Double.MAX_VALUE; for (v : doc['gvalue'].values){ retval = min(v.lon, retval) }; retval")
+                .addScriptField("min", "retval = Double.MAX_VALUE; for (v in doc['gvalue'].values){ retval = min(v.lon, retval) }; retval")
                 .addSort("ord", SortOrder.ASC).setSize(10)
                 .execute().actionGet();