elasticsearch/docs/reference/migration/migrate_5_0/scripting.asciidoc

[[breaking_50_scripting]]
=== Script related changes

==== Switched Default Language from Groovy to Painless

The default scripting language for Elasticsearch is now Painless.  Painless is a custom-built language with syntax
similar to Groovy designed to be fast as well as secure.  Many Groovy scripts will be identitical to Painless scripts
to help make the transition between languages as simple as possible.

Documentation for Painless can be found at <<modules-scripting-painless,Painless Scripting Language>>

It is also possible to set the default language back to Groovy using the following setting: `script.default_lang: groovy`

One common difference to note between Groovy and Painless is the use of parameters -- all parameters in Painless
must be prefixed with `params.` now.  The following example shows the difference:

Groovy:

[source,js]
-----------------------------------
{
  "script_score": {
    "script": {
      "lang": "groovy",
      "inline": "Math.log(_score * 2) + my_modifier",
      "params": {
        "my_modifier": 8
      }
    }
  }
}
-----------------------------------

Painless (`my_modifer` is prefixed with `params`):

[source,js]
-----------------------------------
{
  "script_score": {
    "script": {
      "lang": "painless",
      "inline": "Math.log(_score * 2) + params.my_modifier",
      "params": {
        "my_modifier": 8
      }
    }
  }
}
-----------------------------------

==== Removed 1.x script and template syntax

The deprecated 1.x syntax of defining inline scripts / templates and referring to file or index base scripts / templates
have been removed.

The `script` and `params` string parameters can no longer be used and instead the `script` object syntax must be used.
This applies for the update api, script sort, `script_score` function, `script` query, `scripted_metric` aggregation and
`script_heuristic` aggregation.

So this usage of inline scripts is no longer allowed:

[source,js]
-----------------------------------
{
  "script_score": {
    "lang": "groovy",
    "script": "Math.log(_score * 2) + my_modifier",
    "params": {
      "my_modifier": 8
    }
  }
}
-----------------------------------

and instead this syntax must be used:

[source,js]
-----------------------------------
{
  "script_score": {
    "script": {
      "lang": "groovy",
      "inline": "Math.log(_score * 2) + my_modifier",
      "params": {
        "my_modifier": 8
      }
    }
  }
}
-----------------------------------

The `script` or `script_file` parameter can no longer be used to refer to file based scripts and templates and instead
`file` must be used.

This usage of referring to file based scripts is no longer valid:

[source,js]
-----------------------------------
{
  "script_score": {
    "script": "calculate-score",
    "params": {
      "my_modifier": 8
    }
  }
}
-----------------------------------

This usage is valid:

[source,js]
-----------------------------------
{
  "script_score": {
    "script": {
      "lang": "groovy",
      "file": "calculate-score",
      "params": {
        "my_modifier": 8
      }
    }
  }
}
-----------------------------------

The `script_id` parameter can no longer be used the refer to indexed based scripts and templates and instead `id` must
be used.

This usage of referring to indexed scripts is no longer valid:

[source,js]
-----------------------------------
{
  "script_score": {
    "script_id": "indexedCalculateScore",
    "params": {
      "my_modifier": 8
    }
  }
}
-----------------------------------

This usage is valid:

[source,js]
-----------------------------------
{
  "script_score": {
    "script": {
      "id": "indexedCalculateScore",
      "lang" : "groovy",
      "params": {
        "my_modifier": 8
      }
    }
  }
}
-----------------------------------

==== Template query

The `query` field in the `template` query can no longer be used.
This 1.x syntax can no longer be used:

[source,js]
-----------------------------------
{
    "query": {
        "template": {
            "query": {"match_{{template}}": {}},
            "params" : {
                "template" : "all"
            }
        }
    }
}
-----------------------------------

and instead the following syntax should be used:

[source,js]
-----------------------------------
{
    "query": {
        "template": {
            "inline": {"match_{{template}}": {}},
            "params" : {
                "template" : "all"
            }
        }
    }
}
-----------------------------------

==== Search templates

The top level `template` field in the search template api has been replaced with consistent template / script object
syntax. This 1.x syntax can no longer be used:

[source,js]
-----------------------------------
{
    "template" : {
        "query": { "match" : { "{{my_field}}" : "{{my_value}}" } },
        "size" : "{{my_size}}"
    },
    "params" : {
        "my_field" : "foo",
        "my_value" : "bar",
        "my_size" : 5
    }
}
-----------------------------------

and instead the following syntax should be used:

[source,js]
-----------------------------------
{
    "inline" : {
        "query": { "match" : { "{{my_field}}" : "{{my_value}}" } },
        "size" : "{{my_size}}"
    },
    "params" : {
        "my_field" : "foo",
        "my_value" : "bar",
        "my_size" : 5
    }
}
-----------------------------------

==== Indexed scripts and templates

Indexed scripts and templates have been replaced by <<modules-scripting-stored-scripts,stored scripts>>
which stores the scripts and templates in the cluster state instead of a dedicate `.scripts` index.

For the size of stored scripts there is a soft limit of 65535 bytes. If scripts exceed that size then
the `script.max_size_in_bytes` setting can be added to elasticsearch.yml to change the soft limit to a higher value.
If scripts are really large, other options like native scripts should be considered.

Previously indexed scripts in the `.scripts` index will not be used any more as
Elasticsearch will now try to fetch the scripts from the cluster state. Upon upgrading
to 5.x the `.scripts` index will remain to exist, so it can be used by a script to migrate
the stored scripts from the `.scripts` index into the cluster state. The current format of the scripts
and templates hasn't been changed, only the 1.x format has been removed.

===== Python migration script

The following Python script can be used to import your indexed scripts into the cluster state
as stored scripts:

[source,python]
-----------------------------------
from elasticsearch import Elasticsearch,helpers

es = Elasticsearch([
	{'host': 'localhost'}
])

for doc in helpers.scan(es, index=".scripts", preserve_order=True):
	es.put_script(lang=doc['_type'], id=doc['_id'], body=doc['_source'])
-----------------------------------

This script makes use of the official Elasticsearch Python client and
therefore you need to make sure that your have installed the client in your
environment. For more information on this please see
https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/index.html[`elasticsearch-py`].

===== Perl migration script

The following Perl script can be used to import your indexed scripts into the cluster state
as stored scripts:

[source,perl]
-----------------------------------
use Search::Elasticsearch;

my $es     = Search::Elasticsearch->new( nodes => 'localhost:9200');
my $scroll = $es->scroll_helper( index => '.scripts', sort => '_doc');

while (my $doc = $scroll->next) {
  $e->put_script(
    lang => $doc->{_type},
    id   => $doc->{_id},
    body => $doc->{_source}
  );
}
-----------------------------------

This script makes use of the official Elasticsearch Perl client and
therefore you need to make sure that your have installed the client in your
environment. For more information on this please see
https://metacpan.org/pod/Search::Elasticsearch[`Search::Elasticsearch`].

===== Verifying script migration

After you have moved the scripts via the provided script or otherwise then you can verify with the following
request if the migration has happened successfully:

[source,js]
-----------------------------------
GET _cluster/state?filter_path=metadata.stored_scripts
-----------------------------------

The response should include all your scripts from the `.scripts` index.
After you have verified that all your scripts have been moved, optionally as a last step,
you can delete the `.scripts` index as Elasticsearch no longer uses it.

==== Indexed scripts Java APIs

All the methods related to interacting with indexed scripts have been removed.
The Java API methods for interacting with stored scripts have been added under `ClusterAdminClient` class.
The sugar methods that used to exist on the indexed scripts API methods don't exist on the methods for
stored scripts. The only way to provide scripts is by using `BytesReference` implementation, if a string needs to be
provided the `BytesArray` class should be used.

==== Scripting engines now register only a single language

Prior to 5.0.0, script engines could register multiple languages. The Javascript
script engine in particular registered both `"lang": "js"` and `"lang":
"javascript"`. Script engines can now only register a single language. All
references to `"lang": "js"` should be changed to `"lang": "javascript"` for
existing users of the lang-javascript plugin.

==== Scripting engines now register only a single extension

Prior to 5.0.0 scripting engines could register multiple extensions. The only
engine doing this was the Javascript engine, which registered "js" and
"javascript". It now only registers the "js" file extension for on-disk scripts.

==== `.javascript` files are no longer supported (use `.js`)

The Javascript engine previously registered "js" and "javascript". It now only
registers the "js" file extension for on-disk scripts.

==== Removed scripting query string parameters from update rest api

The `script`, `script_id` and `scripting_upsert` query string parameters have been removed from the update api.

==== Java transport client

The `TemplateQueryBuilder` has been moved to the `lang-mustache` module.
Therefor when using the `TemplateQueryBuilder` from the Java native client the
lang-mustache module should be on the classpath. Also the transport client
should load the lang-mustache module as plugin:

[source,java]
--------------------------------------------------
TransportClient transportClient = TransportClient.builder()
        .settings(Settings.builder().put("node.name", "node"))
        .addPlugin(MustachePlugin.class)
        .build();
transportClient.addTransportAddress(
        new InetSocketTransportAddress(new InetSocketAddress(InetAddresses.forString("127.0.0.1"), 9300))
);
--------------------------------------------------

Also the helper methods in `QueryBuilders` class that create a `TemplateQueryBuilder` instance have been removed,
instead the constructors on `TemplateQueryBuilder` should be used.

==== Template query

The `template` query has been deprecated in favour of the search template api. The `template` query is scheduled
to be removed in the next major version.

==== GeoPoint scripts

The following helper methods have been removed from GeoPoint scripting:

* `factorDistance`
* `factorDistanceWithDefault`
* `factorDistance02`
* `factorDistance13`
* `arcDistanceInKm`
* `arcDistanceInKmWithDefault`
* `arcDistanceInMiles`
* `arcDistanceInMilesWithDefault`
* `distanceWithDefault`
* `distanceInKm`
* `distanceInKmWithDefault`
* `distanceInMiles`
* `distanceInMilesWithDefault`
* `geohashDistanceInKm`
* `geohashDistanceInMiles`

Instead use `arcDistance`, `arcDistanceWithDefault`, `planeDistance`, `planeDistanceWithDefault`, `geohashDistance`,
`geohashDistanceWithDefault` and convert from default units (meters) to desired units using the appropriate constance
(e.g., multiply by `0.001` to convert to Km).