Add Debug.explain to painless

You can use `Debug.explain(someObject)` in painless to throw an
`Error` that can't be caught by painless code and contains an
object's class. This is useful because painless's sandbox doesn't
allow you to call `someObject.getClass()`.

Closes #20263

docs/reference/modules/scripting.asciidoc

@@ -85,6 +85,8 @@ include::scripting/painless.asciidoc[]
 
 include::scripting/painless-syntax.asciidoc[]
 
+include::scripting/painless-debugging.asciidoc[]
+
 include::scripting/expression.asciidoc[]
 
 include::scripting/native.asciidoc[]

docs/reference/modules/scripting/painless-debugging.asciidoc

@@ -0,0 +1,108 @@
+[[modules-scripting-painless-debugging]]
+=== Painless Debugging
+
+experimental[The Painless scripting language is new and is still marked as experimental. The syntax or API may be changed in the future in non-backwards compatible ways if required.]
+
+==== Debug.Explain
+
+Painless doesn't have a
+https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop[REPL]
+and while it'd be nice for it to have one one day, it wouldn't tell you the
+whole story around debugging painless scripts embedded in Elasticsearch because
+the data that the scripts have access to or "context" is so important. For now
+the best way to debug embedded scripts is by throwing exceptions at choice
+places. While you can throw your own exceptions
+(`throw new Exception('whatever')`), Painless's sandbox prevents you from
+accessing useful information like the type of an object. So Painless has a
+utility method, `Debug.explain` which throws the exception for you. For
+example, you can use the <<search-explain>> to explore the context available to
+a <<query-dsl-script-query>>.
+
+[source,js]
+---------------------------------------------------------
+PUT /hockey/player/1?refresh
+{"first":"johnny","last":"gaudreau","goals":[9,27,1],"assists":[17,46,0],"gp":[26,82,1]}
+
+POST /hockey/player/1/_explain
+{
+  "query": {
+    "script": {
+      "script": "Debug.explain(doc.goals)"
+    }
+  }
+}
+---------------------------------------------------------
+// CONSOLE
+// TEST[catch:/painless_explain_error/]
+
+Which shows that the class of `doc.first` is
+`org.elasticsearch.index.fielddata.ScriptDocValues$Longs` by responding with:
+
+[source,js]
+---------------------------------------------------------
+{
+   "error": {
+      "type": "script_exception",
+      "class": "org.elasticsearch.index.fielddata.ScriptDocValues$Longs",
+      "to_string": "[1, 9, 27]",
+      ...
+   },
+   "status": 500
+}
+---------------------------------------------------------
+// TESTRESPONSE[s/\.\.\./"script_stack": $body.error.script_stack, "script": $body.error.script, "lang": $body.error.lang, "caused_by": $body.error.caused_by, "root_cause": $body.error.root_cause, "reason": $body.error.reason/]
+
+You can use the same trick to see that `_source` is a `java.util.LinkedHashMap`
+in the `_update` API:
+
+[source,js]
+---------------------------------------------------------
+POST /hockey/player/1/_update
+{
+  "script": "Debug.explain(ctx._source)"
+}
+---------------------------------------------------------
+// CONSOLE
+// TEST[continued catch:/painless_explain_error/]
+
+The response looks like:
+
+[source,js]
+---------------------------------------------------------
+{
+  "error" : {
+    "root_cause": ...,
+    "type": "illegal_argument_exception",
+    "reason": "failed to execute script",
+    "caused_by": {
+      "type": "script_exception",
+      "class": "java.util.LinkedHashMap",
+      "to_string": "{gp=[26, 82, 1], last=gaudreau, assists=[17, 46, 0], first=johnny, goals=[9, 27, 1]}",
+      ...
+    }
+  },
+  "status": 400
+}
+---------------------------------------------------------
+// TESTRESPONSE[s/"root_cause": \.\.\./"root_cause": $body.error.root_cause/]
+// TESTRESPONSE[s/\.\.\./"script_stack": $body.error.caused_by.script_stack, "script": $body.error.caused_by.script, "lang": $body.error.caused_by.lang, "caused_by": $body.error.caused_by.caused_by, "reason": $body.error.caused_by.reason/]
+// TESTRESPONSE[s/"to_string": ".+"/"to_string": $body.error.caused_by.to_string/]
+
+<1> This is the explanation.
+
+// TODO we should build some javadoc like mashup so people don't have to jump through these hoops.
+
+Once you have the class of an object you can go
+https://github.com/elastic/elasticsearch/tree/{branch}/modules/lang-painless/src/main/resources/org/elasticsearch/painless[here]
+and check the available methods. Painless uses a strict whitelist to prevent
+scripts that don't work well with Elasticsearch and all whitelisted methods
+are listed in a file named after the package of the object (everything before
+the last `.`). So `java.util.Map` is listed in a file named `java.util.txt`
+starting on the line that looks like `class Map -> java.util.Map {`.
+
+With the list of whitelisted methods in hand you can turn to either
+https://docs.oracle.com/javase/8/docs/api/[Javadoc],
+https://github.com/elastic/elasticsearch/tree/{branch}[Elasticsearch's source tree]
+or, for whitelisted methods ending in `*`, the
+https://github.com/elastic/elasticsearch/blob/{branch}/modules/lang-painless/src/main/java/org/elasticsearch/painless/Augmentation.java[Augmentation]
+class.

modules/lang-painless/src/main/java/org/elasticsearch/painless/Debug.java

@@ -0,0 +1,69 @@
+/*
+ * Licensed to Elasticsearch under one or more contributor
+ * license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright
+ * ownership. Elasticsearch licenses this file to you under
+ * the Apache License, Version 2.0 (the "License"); you may
+ * not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.elasticsearch.painless;
+
+import org.elasticsearch.script.ScriptException;
+
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.TreeMap;
+
+import static java.util.Collections.singletonList;
+
+/**
+ * Utility methods for debugging painless scripts that are accessible to painless scripts.
+ */
+public class Debug {
+    private Debug() {}
+
+    /**
+     * Throw an {@link Error} that "explains" an object.
+     */
+    public static void explain(Object objectToExplain) throws PainlessExplainError {
+        throw new PainlessExplainError(objectToExplain);
+    }
+
+    /**
+     * Thrown by {@link Debug#explain(Object)} to explain an object. Subclass of {@linkplain Error} so it cannot be caught by painless
+     * scripts.
+     */
+    public static class PainlessExplainError extends Error {
+        private final Object objectToExplain;
+
+        public PainlessExplainError(Object objectToExplain) {
+            this.objectToExplain = objectToExplain;
+        }
+
+        Object getObjectToExplain() {
+            return objectToExplain;
+        }
+
+        /**
+         * Headers to be added to the {@link ScriptException} for structured rendering.
+         */
+        Map<String, List<String>> getHeaders() {
+            Map<String, List<String>> headers = new TreeMap<>();
+            headers.put("es.class", singletonList(objectToExplain == null ? "null" : objectToExplain.getClass().getName()));
+            headers.put("es.to_string", singletonList(Objects.toString(objectToExplain)));
+            return headers;
+        }
+    }
+}

modules/lang-painless/src/main/java/org/elasticsearch/painless/ScriptImpl.java

@@ -31,6 +31,8 @@ import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
 
+import static java.util.Collections.emptyMap;
+
 /**
  * ScriptImpl can be used as either an {@link ExecutableScript} or a {@link LeafSearchScript}
  * to run a previously compiled Painless script.
@@ -120,18 +122,20 @@ final class ScriptImpl implements ExecutableScript, LeafSearchScript {
         try {
             return executable.execute(variables, scorer, doc, aggregationValue);
         // Note that it is safe to catch any of the following errors since Painless is stateless.
+        } catch (Debug.PainlessExplainError e) {
+            throw convertToScriptException(e, e.getHeaders());
         } catch (PainlessError | BootstrapMethodError | OutOfMemoryError | StackOverflowError | Exception e) {
-            throw convertToScriptException(e);
+            throw convertToScriptException(e, emptyMap());
         }
     }
 
     /**
-     * Adds stack trace and other useful information to exceptiosn thrown
+     * Adds stack trace and other useful information to exceptions thrown
      * from a Painless script.
      * @param t The throwable to build an exception around.
      * @return The generated ScriptException.
      */
-    private ScriptException convertToScriptException(Throwable t) {
+    private ScriptException convertToScriptException(Throwable t, Map<String, List<String>> headers) {
         // create a script stack: this is just the script portion
         List<String> scriptStack = new ArrayList<>();
         for (StackTraceElement element : t.getStackTrace()) {
@@ -174,7 +178,11 @@ final class ScriptImpl implements ExecutableScript, LeafSearchScript {
         } else {
             name = executable.getName();
         }
-        throw new ScriptException("runtime error", t, scriptStack, name, PainlessScriptEngineService.NAME);
+        ScriptException scriptException = new ScriptException("runtime error", t, scriptStack, name, PainlessScriptEngineService.NAME);
+        for (Map.Entry<String, List<String>> header : headers.entrySet()) {
+            scriptException.addHeader(header.getKey(), header.getValue());
+        }
+        return scriptException;
     }
 
     /** returns true for methods that are part of the runtime */

modules/lang-painless/src/main/resources/org/elasticsearch/painless/org.elasticsearch.txt

@@ -57,6 +57,13 @@ class def -> java.lang.Object {
   String toString()
 }
 
+
+#### Painless debugging API
+
+class Debug -> org.elasticsearch.painless.Debug extends Object {
+  void explain(Object)
+}
+
 #### ES Scripting API
 
 class org.elasticsearch.common.geo.GeoPoint -> org.elasticsearch.common.geo.GeoPoint extends Object {

modules/lang-painless/src/test/java/org/elasticsearch/painless/DebugTests.java

@@ -0,0 +1,79 @@
+/*
+ * Licensed to Elasticsearch under one or more contributor
+ * license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright
+ * ownership. Elasticsearch licenses this file to you under
+ * the Apache License, Version 2.0 (the "License"); you may
+ * not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.elasticsearch.painless;
+
+import org.elasticsearch.ElasticsearchException;
+import org.elasticsearch.common.io.stream.BytesStreamOutput;
+import org.elasticsearch.common.io.stream.StreamInput;
+import org.elasticsearch.script.ScriptException;
+
+import java.io.IOException;
+import java.util.Map;
+
+import static java.util.Collections.singletonList;
+import static java.util.Collections.singletonMap;
+import static org.hamcrest.Matchers.hasEntry;
+
+public class DebugTests extends ScriptTestCase {
+    public void testExplain() {
+        Object dummy = new Object();
+        Map<String, Object> params = singletonMap("a", dummy);
+
+        Debug.PainlessExplainError e = expectScriptThrows(Debug.PainlessExplainError.class, () -> exec(
+                "Debug.explain(params.a)", params, true));
+        assertSame(dummy, e.getObjectToExplain());
+        assertThat(e.getHeaders(), hasEntry("es.class", singletonList("java.lang.Object")));
+        assertThat(e.getHeaders(), hasEntry("es.to_string", singletonList(dummy.toString())));
+
+        // Null should be ok
+        e = expectScriptThrows(Debug.PainlessExplainError.class, () -> exec("Debug.explain(null)"));
+        assertNull(e.getObjectToExplain());
+        assertThat(e.getHeaders(), hasEntry("es.class", singletonList("null")));
+        assertThat(e.getHeaders(), hasEntry("es.to_string", singletonList("null")));
+
+        // You can't catch the explain exception
+        e = expectScriptThrows(Debug.PainlessExplainError.class, () -> exec(
+                "try {\n"
+              + "  Debug.explain(params.a)\n"
+              + "} catch (Exception e) {\n"
+              + "  return 1\n"
+              + "}", params, true));
+        assertSame(dummy, e.getObjectToExplain());
+    }
+
+    /**
+     * {@link Debug.PainlessExplainError} doesn't serialize but the headers still make it.
+     */
+    public void testPainlessExplainErrorSerialization() throws IOException {
+        Map<String, Object> params = singletonMap("a", "jumped over the moon");
+        ScriptException e = expectThrows(ScriptException.class, () -> exec("Debug.explain(params.a)", params, true));
+        assertEquals(singletonList("java.lang.String"), e.getHeader("es.class"));
+        assertEquals(singletonList("jumped over the moon"), e.getHeader("es.to_string"));
+
+        try (BytesStreamOutput out = new BytesStreamOutput()) {
+            out.writeException(e);
+            try (StreamInput in = out.bytes().streamInput()) {
+                ElasticsearchException read = (ScriptException) in.readException();
+                assertEquals(singletonList("java.lang.String"), read.getHeader("es.class"));
+                assertEquals(singletonList("jumped over the moon"), read.getHeader("es.to_string"));
+            }
+        }
+    }
+}