[SPATIAL] New ShapeQueryBuilder for querying indexed cartesian geometry (#45108)

This commit adds a new ShapeQueryBuilder to the xpack spatial module for
querying arbitrary Cartesian geometries indexed using the new shape field
type.

The query builder extends AbstractGeometryQueryBuilder and leverages the
ShapeQueryProcessor added in the previous field mapper commit.

Tests are provided in ShapeQueryTests in the same manner as
GeoShapeQueryTests and docs are updated to explain how the query works.

docs/reference/mapping/types/shape.asciidoc

@@ -11,6 +11,9 @@ with arbitrary `x, y` cartesian shapes such as rectangles and polygons. It can b
 used to index and query geometries whose coordinates fall in a 2-dimensional planar
 coordinate system.
 
+You can query documents using this type using
+<<query-dsl-shape-query,shape Query>>.
+
 [[shape-mapping-options]]
 [float]
 ==== Mapping Options

docs/reference/query-dsl.asciidoc

@@ -35,6 +35,8 @@ include::query-dsl/full-text-queries.asciidoc[]
 
 include::query-dsl/geo-queries.asciidoc[]
 
+include::query-dsl/shape-queries.asciidoc[]
+
 include::query-dsl/joining-queries.asciidoc[]
 
 include::query-dsl/match-all-query.asciidoc[]

docs/reference/query-dsl/shape-queries.asciidoc

@@ -0,0 +1,18 @@
+[[shape-queries]]
+[role="xpack"]
+[testenv="basic"]
+== Shape queries
+
+Like <<geo-shape,`geo_shape`>> Elasticsearch supports the ability to index
+arbitrary two dimension (non Geospatial) geometries making it possible to
+map out virtual worlds, sporting venues, theme parks, and CAD diagrams. The
+<<shape,`shape`>> field type supports points, lines, polygons, multi-polygons,
+envelope, etc.
+
+The queries in this group are:
+
+<<query-dsl-shape-query,`shape`>> query::
+Finds documents with shapes that either intersect, are within, or do not
+intersect a specified shape.
+
+include::shape-query.asciidoc[]

docs/reference/query-dsl/shape-query.asciidoc

@@ -0,0 +1,149 @@
+[[query-dsl-shape-query]]
+[role="xpack"]
+[testenv="basic"]
+=== Shape query
+++++
+<titleabbrev>Shape</titleabbrev>
+++++
+
+Queries documents that contain fields indexed using the `shape` type.
+
+Requires the <<shape,`shape` Mapping>>.
+
+The query supports two ways of defining the target shape, either by
+providing a whole shape definition, or by referencing the name, or id, of a shape
+pre-indexed in another index. Both formats are defined below with
+examples.
+
+==== Inline Shape Definition
+
+Similar to the `geo_shape` query, the `shape` query uses
+http://www.geojson.org[GeoJSON] or
+https://en.wikipedia.org/wiki/Well-known_text_representation_of_geometry[Well Known Text]
+(WKT) to represent shapes.
+
+Given the following index:
+
+[source,js]
+--------------------------------------------------
+PUT /example
+{
+    "mappings": {
+        "properties": {
+            "geometry": {
+                "type": "shape"
+            }
+        }
+    }
+}
+
+POST /example/_doc?refresh
+{
+    "name": "Lucky Landing",
+    "location": {
+        "type": "point",
+        "coordinates": [1355.400544, 5255.530286]
+    }
+}
+--------------------------------------------------
+// CONSOLE
+// TESTSETUP
+
+The following query will find the point using the Elasticsearch's
+`envelope` GeoJSON extension:
+
+[source,js]
+--------------------------------------------------
+GET /example/_search
+{
+    "query":{
+        "shape": {
+            "geometry": {
+                "shape": {
+                    "type": "envelope",
+                    "coordinates" : [[1355.0, 5355.0], [1400.0, 5200.0]]
+                },
+                "relation": "within"
+            }
+        }
+    }
+}
+--------------------------------------------------
+// CONSOLE
+
+==== Pre-Indexed Shape
+
+The Query also supports using a shape which has already been indexed in
+another index. This is particularly useful for when
+you have a pre-defined list of shapes which are useful to your
+application and you want to reference this using a logical name (for
+example 'New Zealand') rather than having to provide their coordinates
+each time. In this situation it is only necessary to provide:
+
+* `id` - The ID of the document that containing the pre-indexed shape.
+* `index` - Name of the index where the pre-indexed shape is. Defaults
+to 'shapes'.
+* `path` - The field specified as path containing the pre-indexed shape.
+Defaults to 'shape'.
+* `routing` - The routing of the shape document if required.
+
+The following is an example of using the Filter with a pre-indexed
+shape:
+
+[source,js]
+--------------------------------------------------
+PUT /shapes
+{
+    "mappings": {
+        "properties": {
+            "geometry": {
+                "type": "shape"
+            }
+        }
+    }
+}
+
+PUT /shapes/_doc/footprint
+{
+    "geometry": {
+        "type": "envelope",
+        "coordinates" : [[1355.0, 5355.0], [1400.0, 5200.0]]
+    }
+}
+
+GET /example/_search
+{
+    "query": {
+        "shape": {
+            "geometry": {
+                "indexed_shape": {
+                    "index": "shapes",
+                    "id": "footprint",
+                    "path": "geometry"
+                }
+            }
+        }
+    }
+}
+--------------------------------------------------
+// CONSOLE
+
+==== Spatial Relations
+
+The following is a complete list of spatial relation operators available:
+
+* `INTERSECTS` - (default) Return all documents whose `geo_shape` field
+intersects the query geometry.
+* `DISJOINT` - Return all documents whose `geo_shape` field
+has nothing in common with the query geometry.
+* `WITHIN` - Return all documents whose `geo_shape` field
+is within the query geometry.
+
+[float]
+==== Ignore Unmapped
+
+When set to `true` the `ignore_unmapped` option will ignore an unmapped field
+and will not match any documents for this query. This can be useful when
+querying multiple indexes which might have different mappings. When set to
+`false` (the default value) the query will throw an exception if the field
+is not mapped.

server/src/main/java/org/elasticsearch/index/query/AbstractGeometryQueryBuilder.java

@@ -544,7 +544,7 @@ public abstract class AbstractGeometryQueryBuilder<QB extends AbstractGeometryQu
     }
 
     /** local class that encapsulates xcontent parsed shape parameters */
-    protected abstract static class ParsedShapeQueryParams {
+    protected abstract static class ParsedGeometryQueryParams {
         public String fieldName;
         public ShapeRelation relation;
         public ShapeBuilder shape;
@@ -562,7 +562,7 @@ public abstract class AbstractGeometryQueryBuilder<QB extends AbstractGeometryQu
         protected abstract boolean parseXContentField(XContentParser parser) throws IOException;
     }
 
-    public static ParsedShapeQueryParams parsedParamsFromXContent(XContentParser parser, ParsedShapeQueryParams params)
+    public static ParsedGeometryQueryParams parsedParamsFromXContent(XContentParser parser, ParsedGeometryQueryParams params)
         throws IOException {
         String fieldName = null;
         XContentParser.Token token;

server/src/main/java/org/elasticsearch/index/query/GeoShapeQueryBuilder.java

@@ -237,7 +237,7 @@ public class GeoShapeQueryBuilder extends AbstractGeometryQueryBuilder<GeoShapeQ
         return builder;
     }
 
-    private static class ParsedGeoShapeQueryParams extends ParsedShapeQueryParams {
+    private static class ParsedGeoShapeQueryParams extends ParsedGeometryQueryParams {
         SpatialStrategy strategy;
 
         @Override

test/framework/src/main/java/org/elasticsearch/geo/GeometryTestUtils.java

@@ -61,13 +61,15 @@ public class GeometryTestUtils {
     }
 
     public static Line randomLine(boolean hasAlts) {
-        int size = ESTestCase.randomIntBetween(2, 10);
+        // we use nextPolygon because it guarantees no duplicate points
+        org.apache.lucene.geo.Polygon lucenePolygon = GeoTestUtil.nextPolygon();
+        int size = lucenePolygon.numPoints() - 1;
         double[] lats = new double[size];
         double[] lons = new double[size];
         double[] alts = hasAlts ? new double[size] : null;
         for (int i = 0; i < size; i++) {
-            lats[i] = randomLat();
-            lons[i] = randomLon();
+            lats[i] = lucenePolygon.getPolyLat(i);
+            lons[i] = lucenePolygon.getPolyLon(i);
             if (hasAlts) {
                 alts[i] = randomAlt();
             }
@@ -96,11 +98,12 @@ public class GeometryTestUtils {
             org.apache.lucene.geo.Polygon[] luceneHoles = lucenePolygon.getHoles();
             List<LinearRing> holes = new ArrayList<>();
             for (int i = 0; i < lucenePolygon.numHoles(); i++) {
-                holes.add(linearRing(luceneHoles[i], hasAlt));
+                org.apache.lucene.geo.Polygon poly = luceneHoles[i];
+                holes.add(linearRing(poly.getPolyLats(), poly.getPolyLons(), hasAlt));
             }
-            return new Polygon(linearRing(lucenePolygon, hasAlt), holes);
+            return new Polygon(linearRing(lucenePolygon.getPolyLats(), lucenePolygon.getPolyLons(), hasAlt), holes);
         }
-        return new Polygon(linearRing(lucenePolygon, hasAlt));
+        return new Polygon(linearRing(lucenePolygon.getPolyLats(), lucenePolygon.getPolyLons(), hasAlt));
     }
 
 
@@ -113,12 +116,11 @@ public class GeometryTestUtils {
         return alts;
     }
 
-    private static LinearRing linearRing(org.apache.lucene.geo.Polygon polygon, boolean generateAlts) {
+    public static LinearRing linearRing(double[] lats, double[] lons, boolean generateAlts) {
         if (generateAlts) {
-            return new LinearRing(polygon.getPolyLats(), polygon.getPolyLons(), randomAltRing(polygon.numPoints()));
-        } else {
-            return new LinearRing(polygon.getPolyLats(), polygon.getPolyLons());
+            return new LinearRing(lats, lons, randomAltRing(lats.length));
         }
+        return new LinearRing(lats, lons);
     }
 
     public static Rectangle randomRectangle() {
@@ -170,7 +172,7 @@ public class GeometryTestUtils {
         return randomGeometry(0, hasAlt);
     }
 
-    private static Geometry randomGeometry(int level, boolean hasAlt) {
+    protected static Geometry randomGeometry(int level, boolean hasAlt) {
         @SuppressWarnings("unchecked") Function<Boolean, Geometry> geometry = ESTestCase.randomFrom(
             GeometryTestUtils::randomCircle,
             GeometryTestUtils::randomLine,

x-pack/plugin/spatial/build.gradle

@@ -17,6 +17,11 @@ dependencies {
     }
 }
 
+licenseHeaders {
+    // This class was sourced from apache lucene's sandbox module tests
+    excludes << 'org/apache/lucene/geo/XShapeTestUtil.java'
+}
+
 // xpack modules are installed in real clusters as the meta plugin, so
 // installing them as individual plugins for integ tests doesn't make sense,
 // so we disable integ tests

x-pack/plugin/spatial/src/main/java/org/elasticsearch/xpack/spatial/SpatialPlugin.java

@@ -12,9 +12,11 @@ import org.elasticsearch.index.mapper.Mapper;
 import org.elasticsearch.plugins.ActionPlugin;
 import org.elasticsearch.plugins.MapperPlugin;
 import org.elasticsearch.plugins.Plugin;
+import org.elasticsearch.plugins.SearchPlugin;
 import org.elasticsearch.xpack.core.action.XPackInfoFeatureAction;
 import org.elasticsearch.xpack.core.action.XPackUsageFeatureAction;
 import org.elasticsearch.xpack.spatial.index.mapper.ShapeFieldMapper;
+import org.elasticsearch.xpack.spatial.index.query.ShapeQueryBuilder;
 
 import java.util.Arrays;
 import java.util.Collections;
@@ -22,7 +24,9 @@ import java.util.LinkedHashMap;
 import java.util.List;
 import java.util.Map;
 
-public class SpatialPlugin extends Plugin implements ActionPlugin, MapperPlugin {
+import static java.util.Collections.singletonList;
+
+public class SpatialPlugin extends Plugin implements ActionPlugin, MapperPlugin, SearchPlugin {
 
     public SpatialPlugin(Settings settings) {
     }
@@ -40,4 +44,9 @@ public class SpatialPlugin extends Plugin implements ActionPlugin, MapperPlugin
         mappers.put(ShapeFieldMapper.CONTENT_TYPE, new ShapeFieldMapper.TypeParser());
         return Collections.unmodifiableMap(mappers);
     }
+
+    @Override
+    public List<QuerySpec<?>> getQueries() {
+        return singletonList(new QuerySpec<>(ShapeQueryBuilder.NAME, ShapeQueryBuilder::new, ShapeQueryBuilder::fromXContent));
+    }
 }

x-pack/plugin/spatial/src/main/java/org/elasticsearch/xpack/spatial/SpatialUsageTransportAction.java

@@ -40,8 +40,7 @@ public class SpatialUsageTransportAction extends XPackUsageFeatureTransportActio
     @Override
     protected void masterOperation(Task task, XPackUsageRequest request, ClusterState state,
                                    ActionListener<XPackUsageFeatureResponse> listener) {
-        SpatialFeatureSetUsage usage =
-            new SpatialFeatureSetUsage(licenseState.isSpatialAllowed(), true);
+        SpatialFeatureSetUsage usage = new SpatialFeatureSetUsage(licenseState.isSpatialAllowed(), true);
         listener.onResponse(new XPackUsageFeatureResponse(usage));
     }
 }

x-pack/plugin/spatial/src/main/java/org/elasticsearch/xpack/spatial/index/query/ShapeQueryBuilder.java

@@ -0,0 +1,213 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License;
+ * you may not use this file except in compliance with the Elastic License.
+ */
+package org.elasticsearch.xpack.spatial.index.query;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.lucene.search.Query;
+import org.elasticsearch.common.Nullable;
+import org.elasticsearch.common.geo.builders.ShapeBuilder;
+import org.elasticsearch.common.geo.parsers.ShapeParser;
+import org.elasticsearch.common.io.stream.StreamInput;
+import org.elasticsearch.common.io.stream.StreamOutput;
+import org.elasticsearch.common.logging.DeprecationLogger;
+import org.elasticsearch.common.xcontent.XContentBuilder;
+import org.elasticsearch.common.xcontent.XContentParser;
+import org.elasticsearch.geo.geometry.Geometry;
+import org.elasticsearch.index.mapper.AbstractGeometryFieldMapper;
+import org.elasticsearch.index.mapper.MappedFieldType;
+import org.elasticsearch.index.query.AbstractGeometryQueryBuilder;
+import org.elasticsearch.index.query.GeoShapeQueryBuilder;
+import org.elasticsearch.index.query.QueryRewriteContext;
+import org.elasticsearch.index.query.QueryShardContext;
+import org.elasticsearch.index.query.QueryShardException;
+import org.elasticsearch.xpack.spatial.index.mapper.ShapeFieldMapper;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+import java.util.function.Supplier;
+
+/**
+ * Derived {@link AbstractGeometryQueryBuilder} that builds a {@code x, y} Shape Query
+ *
+ * GeoJson and WKT shape definitions are supported
+ */
+public class ShapeQueryBuilder extends AbstractGeometryQueryBuilder<ShapeQueryBuilder> {
+    public static final String NAME = "shape";
+
+    private static final DeprecationLogger deprecationLogger = new DeprecationLogger(
+        LogManager.getLogger(GeoShapeQueryBuilder.class));
+
+    static final String TYPES_DEPRECATION_MESSAGE = "[types removal] Types are deprecated in [geo_shape] queries. " +
+        "The type should no longer be specified in the [indexed_shape] section.";
+
+    /**
+     * Creates a new GeoShapeQueryBuilder whose Query will be against the given
+     * field name using the given Shape
+     *
+     * @param fieldName
+     *            Name of the field that will be queried
+     * @param shape
+     *            Shape used in the Query
+     * @deprecated use {@link #ShapeQueryBuilder(String, Geometry)} instead
+     */
+    @Deprecated
+    @SuppressWarnings({ "rawtypes" })
+    protected ShapeQueryBuilder(String fieldName, ShapeBuilder shape) {
+        super(fieldName, shape);
+    }
+
+    /**
+     * Creates a new GeoShapeQueryBuilder whose Query will be against the given
+     * field name using the given Shape
+     *
+     * @param fieldName
+     *            Name of the field that will be queried
+     * @param shape
+     *            Shape used in the Query
+     */
+    public ShapeQueryBuilder(String fieldName, Geometry shape) {
+        super(fieldName, shape);
+    }
+
+    protected ShapeQueryBuilder(String fieldName, Supplier<Geometry> shapeSupplier, String indexedShapeId,
+                                @Nullable String indexedShapeType) {
+        super(fieldName, shapeSupplier, indexedShapeId, indexedShapeType);
+    }
+
+    /**
+     * Creates a new GeoShapeQueryBuilder whose Query will be against the given
+     * field name and will use the Shape found with the given ID
+     *
+     * @param fieldName
+     *            Name of the field that will be filtered
+     * @param indexedShapeId
+     *            ID of the indexed Shape that will be used in the Query
+     */
+    public ShapeQueryBuilder(String fieldName, String indexedShapeId) {
+        super(fieldName, indexedShapeId);
+    }
+
+    @Deprecated
+    protected ShapeQueryBuilder(String fieldName, String indexedShapeId, String indexedShapeType) {
+        super(fieldName, (Geometry) null, indexedShapeId, indexedShapeType);
+    }
+
+    public ShapeQueryBuilder(StreamInput in) throws IOException {
+        super(in);
+    }
+
+    @Override
+    protected void doWriteTo(StreamOutput out) throws IOException {
+        super.doWriteTo(out);
+    }
+
+    @Override
+    protected ShapeQueryBuilder newShapeQueryBuilder(String fieldName, Geometry shape) {
+        return new ShapeQueryBuilder(fieldName, shape);
+    }
+
+    @Override
+    protected ShapeQueryBuilder newShapeQueryBuilder(String fieldName, Supplier<Geometry> shapeSupplier, String indexedShapeId,
+                                                     String indexedShapeType) {
+        return new ShapeQueryBuilder(fieldName, shapeSupplier, indexedShapeId, indexedShapeType);
+    }
+
+    @Override
+    public String queryFieldType() {
+        return ShapeFieldMapper.CONTENT_TYPE;
+    }
+
+    @Override
+    @SuppressWarnings({ "rawtypes" })
+    protected List validContentTypes() {
+        return Arrays.asList(ShapeFieldMapper.CONTENT_TYPE);
+    }
+
+    @Override
+    @SuppressWarnings({ "rawtypes" })
+    public Query buildShapeQuery(QueryShardContext context, MappedFieldType fieldType) {
+        if (fieldType.typeName().equals(ShapeFieldMapper.CONTENT_TYPE) == false) {
+            throw new QueryShardException(context,
+                "Field [" + fieldName + "] is not of type [" + queryFieldType() + "] but of type [" + fieldType.typeName() + "]");
+        }
+
+        final AbstractGeometryFieldMapper.AbstractGeometryFieldType ft = (AbstractGeometryFieldMapper.AbstractGeometryFieldType) fieldType;
+        return ft.geometryQueryBuilder().process(shape, ft.name(), relation, context);
+    }
+
+    @Override
+    public void doShapeQueryXContent(XContentBuilder builder, Params params) throws IOException {
+        // noop
+    }
+
+    @Override
+    protected ShapeQueryBuilder doRewrite(QueryRewriteContext queryRewriteContext) throws IOException {
+        return (ShapeQueryBuilder)super.doRewrite(queryRewriteContext);
+    }
+
+    @Override
+    protected boolean doEquals(ShapeQueryBuilder other) {
+        return super.doEquals((AbstractGeometryQueryBuilder)other);
+    }
+
+    @Override
+    protected int doHashCode() {
+        return Objects.hash(super.doHashCode());
+    }
+
+    @Override
+    public String getWriteableName() {
+        return NAME;
+    }
+
+    private static class ParsedShapeQueryParams extends ParsedGeometryQueryParams {
+        @Override
+        protected boolean parseXContentField(XContentParser parser) throws IOException {
+            if (SHAPE_FIELD.match(parser.currentName(), parser.getDeprecationHandler())) {
+                this.shape = ShapeParser.parse(parser);
+                return true;
+            }
+            return false;
+        }
+    }
+
+    public static ShapeQueryBuilder fromXContent(XContentParser parser) throws IOException {
+        ParsedShapeQueryParams pgsqb = (ParsedShapeQueryParams)AbstractGeometryQueryBuilder.parsedParamsFromXContent(parser,
+            new ParsedShapeQueryParams());
+
+        ShapeQueryBuilder builder;
+        if (pgsqb.type != null) {
+            deprecationLogger.deprecatedAndMaybeLog(
+                "geo_share_query_with_types", TYPES_DEPRECATION_MESSAGE);
+        }
+
+        if (pgsqb.shape != null) {
+            builder = new ShapeQueryBuilder(pgsqb.fieldName, pgsqb.shape);
+        } else {
+            builder = new ShapeQueryBuilder(pgsqb.fieldName, pgsqb.id, pgsqb.type);
+        }
+        if (pgsqb.index != null) {
+            builder.indexedShapeIndex(pgsqb.index);
+        }
+        if (pgsqb.shapePath != null) {
+            builder.indexedShapePath(pgsqb.shapePath);
+        }
+        if (pgsqb.shapeRouting != null) {
+            builder.indexedShapeRouting(pgsqb.shapeRouting);
+        }
+        if (pgsqb.relation != null) {
+            builder.relation(pgsqb.relation);
+        }
+        if (pgsqb.queryName != null) {
+            builder.queryName(pgsqb.queryName);
+        }
+        builder.boost(pgsqb.boost);
+        builder.ignoreUnmapped(pgsqb.ignoreUnmapped);
+        return builder;
+    }
+}

x-pack/plugin/spatial/src/test/java/org/apache/lucene/geo/XShapeTestUtil.java

@@ -0,0 +1,208 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.geo;
+
+import java.util.ArrayList;
+import java.util.Random;
+
+import com.carrotsearch.randomizedtesting.RandomizedContext;
+import com.carrotsearch.randomizedtesting.generators.BiasedNumbers;
+import org.apache.lucene.util.SloppyMath;
+import org.apache.lucene.util.TestUtil;
+
+/** generates random cartesian geometry; heavy reuse of {@link GeoTestUtil} */
+public class XShapeTestUtil {
+
+    /** returns next pseudorandom polygon */
+    public static XYPolygon nextPolygon() {
+        if (random().nextBoolean()) {
+            return surpriseMePolygon();
+        } else if (random().nextInt(10) == 1) {
+            // this poly is slow to create ... only do it 10% of the time:
+            while (true) {
+                int gons = TestUtil.nextInt(random(), 4, 500);
+                // So the poly can cover at most 50% of the earth's surface:
+                double radius = random().nextDouble() * 0.5 * Float.MAX_VALUE + 1.0;
+                try {
+                    return createRegularPolygon(nextDouble(), nextDouble(), radius, gons);
+                } catch (IllegalArgumentException iae) {
+                    // we tried to cross dateline or pole ... try again
+                }
+            }
+        }
+
+        XYRectangle box = nextBoxInternal();
+        if (random().nextBoolean()) {
+            // box
+            return boxPolygon(box);
+        } else {
+            // triangle
+            return trianglePolygon(box);
+        }
+    }
+
+    private static XYPolygon trianglePolygon(XYRectangle box) {
+        final float[] polyX = new float[4];
+        final float[] polyY = new float[4];
+        polyX[0] = (float)box.minX;
+        polyY[0] = (float)box.minY;
+        polyX[1] = (float)box.minX;
+        polyY[1] = (float)box.minY;
+        polyX[2] = (float)box.minX;
+        polyY[2] = (float)box.minY;
+        polyX[3] = (float)box.minX;
+        polyY[3] = (float)box.minY;
+        return new XYPolygon(polyX, polyY);
+    }
+
+    public static XYRectangle nextBox() {
+        return nextBoxInternal();
+    }
+
+    private static XYRectangle nextBoxInternal() {
+        // prevent lines instead of boxes
+        double x0 = nextDouble();
+        double x1 = nextDouble();
+        while (x0 == x1) {
+            x1 = nextDouble();
+        }
+        // prevent lines instead of boxes
+        double y0 = nextDouble();
+        double y1 = nextDouble();
+        while (y0 == y1) {
+            y1 = nextDouble();
+        }
+
+        if (x1 < x0) {
+            double x = x0;
+            x0 = x1;
+            x1 = x;
+        }
+
+        if (y1 < y0) {
+            double y = y0;
+            y0 = y1;
+            y1 = y;
+        }
+
+        return new XYRectangle(x0, x1, y0, y1);
+    }
+
+    private static XYPolygon boxPolygon(XYRectangle box) {
+        final float[] polyX = new float[5];
+        final float[] polyY = new float[5];
+        polyX[0] = (float)box.minX;
+        polyY[0] = (float)box.minY;
+        polyX[1] = (float)box.minX;
+        polyY[1] = (float)box.minY;
+        polyX[2] = (float)box.minX;
+        polyY[2] = (float)box.minY;
+        polyX[3] = (float)box.minX;
+        polyY[3] = (float)box.minY;
+        polyX[4] = (float)box.minX;
+        polyY[4] = (float)box.minY;
+        return new XYPolygon(polyX, polyY);
+    }
+
+    private static XYPolygon surpriseMePolygon() {
+        // repeat until we get a poly that doesn't cross dateline:
+        while (true) {
+            //System.out.println("\nPOLY ITER");
+            double centerX = nextDouble();
+            double centerY = nextDouble();
+            double radius = 0.1 + 20 * random().nextDouble();
+            double radiusDelta = random().nextDouble();
+
+            ArrayList<Float> xList = new ArrayList<>();
+            ArrayList<Float> yList = new ArrayList<>();
+            double angle = 0.0;
+            while (true) {
+                angle += random().nextDouble()*40.0;
+                //System.out.println("  angle " + angle);
+                if (angle > 360) {
+                    break;
+                }
+                double len = radius * (1.0 - radiusDelta + radiusDelta * random().nextDouble());
+                double maxX = StrictMath.min(StrictMath.abs(Float.MAX_VALUE - centerX), StrictMath.abs(-Float.MAX_VALUE - centerX));
+                double maxY = StrictMath.min(StrictMath.abs(Float.MAX_VALUE - centerY), StrictMath.abs(-Float.MAX_VALUE - centerY));
+
+                len = StrictMath.min(len, StrictMath.min(maxX, maxY));
+
+                //System.out.println("    len=" + len);
+                float x = (float)(centerX + len * Math.cos(SloppyMath.toRadians(angle)));
+                float y = (float)(centerY + len * Math.sin(SloppyMath.toRadians(angle)));
+
+                xList.add(x);
+                yList.add(y);
+
+                //System.out.println("    lat=" + lats.get(lats.size()-1) + " lon=" + lons.get(lons.size()-1));
+            }
+
+            // close it
+            xList.add(xList.get(0));
+            yList.add(yList.get(0));
+
+            float[] xArray = new float[xList.size()];
+            float[] yArray = new float[yList.size()];
+            for(int i=0;i<xList.size();i++) {
+                xArray[i] = xList.get(i);
+                yArray[i] = yList.get(i);
+            }
+            return new XYPolygon(xArray, yArray);
+        }
+    }
+
+    /** Makes an n-gon, centered at the provided x/y, and each vertex approximately
+     *  distanceMeters away from the center.
+     *
+     * Do not invoke me across the dateline or a pole!! */
+    public static XYPolygon createRegularPolygon(double centerX, double centerY, double radius, int gons) {
+
+        double maxX = StrictMath.min(StrictMath.abs(Float.MAX_VALUE - centerX), StrictMath.abs(-Float.MAX_VALUE - centerX));
+        double maxY = StrictMath.min(StrictMath.abs(Float.MAX_VALUE - centerY), StrictMath.abs(-Float.MAX_VALUE - centerY));
+
+        radius = StrictMath.min(radius, StrictMath.min(maxX, maxY));
+
+        float[][] result = new float[2][];
+        result[0] = new float[gons+1];
+        result[1] = new float[gons+1];
+        //System.out.println("make gon=" + gons);
+        for(int i=0;i<gons;i++) {
+            double angle = 360.0-i*(360.0/gons);
+            //System.out.println("  angle " + angle);
+            double x = Math.cos(StrictMath.toRadians(angle));
+            double y = Math.sin(StrictMath.toRadians(angle));
+            result[0][i] = (float)(centerY + y * radius);
+            result[1][i] = (float)(centerX + x * radius);
+        }
+
+        // close poly
+        result[0][gons] = result[0][0];
+        result[1][gons] = result[1][0];
+
+        return new XYPolygon(result[0], result[1]);
+    }
+
+    public static double nextDouble() {
+        return BiasedNumbers.randomDoubleBetween(random(), -Float.MAX_VALUE, Float.MAX_VALUE);
+    }
+
+    /** Keep it simple, we don't need to take arbitrary Random for geo tests */
+    private static Random random() {
+        return RandomizedContext.current().getRandom();
+    }
+}

x-pack/plugin/spatial/src/test/java/org/elasticsearch/xpack/spatial/index/query/ShapeQueryBuilderTests.java

@@ -0,0 +1,291 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License;
+ * you may not use this file except in compliance with the Elastic License.
+ */
+package org.elasticsearch.xpack.spatial.index.query;
+
+import org.apache.lucene.search.BooleanQuery;
+import org.apache.lucene.search.ConstantScoreQuery;
+import org.apache.lucene.search.MatchNoDocsQuery;
+import org.apache.lucene.search.Query;
+import org.elasticsearch.ElasticsearchException;
+import org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest;
+import org.elasticsearch.action.get.GetRequest;
+import org.elasticsearch.action.get.GetResponse;
+import org.elasticsearch.common.Strings;
+import org.elasticsearch.common.bytes.BytesArray;
+import org.elasticsearch.common.compress.CompressedXContent;
+import org.elasticsearch.common.geo.GeoJson;
+import org.elasticsearch.common.geo.ShapeRelation;
+import org.elasticsearch.common.io.stream.BytesStreamOutput;
+import org.elasticsearch.common.xcontent.ToXContentObject;
+import org.elasticsearch.common.xcontent.XContentBuilder;
+import org.elasticsearch.common.xcontent.XContentFactory;
+import org.elasticsearch.common.xcontent.XContentParser;
+import org.elasticsearch.geo.geometry.Geometry;
+import org.elasticsearch.geo.geometry.ShapeType;
+import org.elasticsearch.index.get.GetResult;
+import org.elasticsearch.index.mapper.MapperService;
+import org.elasticsearch.index.query.BoolQueryBuilder;
+import org.elasticsearch.index.query.QueryBuilder;
+import org.elasticsearch.index.query.QueryShardException;
+import org.elasticsearch.index.query.Rewriteable;
+import org.elasticsearch.plugins.Plugin;
+import org.elasticsearch.search.internal.SearchContext;
+import org.elasticsearch.test.AbstractQueryTestCase;
+import org.elasticsearch.xpack.spatial.SpatialPlugin;
+import org.elasticsearch.xpack.spatial.util.ShapeTestUtils;
+import org.junit.After;
+
+import java.io.IOException;
+import java.util.Collection;
+import java.util.Collections;
+
+import static org.hamcrest.CoreMatchers.containsString;
+import static org.hamcrest.CoreMatchers.instanceOf;
+import static org.hamcrest.CoreMatchers.notNullValue;
+import static org.hamcrest.Matchers.anyOf;
+import static org.hamcrest.Matchers.equalTo;
+
+public class ShapeQueryBuilderTests extends AbstractQueryTestCase<ShapeQueryBuilder> {
+
+    protected static final String SHAPE_FIELD_NAME = "mapped_shape";
+
+    private static String docType = "_doc";
+
+    protected static String indexedShapeId;
+    protected static String indexedShapeType;
+    protected static String indexedShapePath;
+    protected static String indexedShapeIndex;
+    protected static String indexedShapeRouting;
+    protected static Geometry indexedShapeToReturn;
+
+    @Override
+    protected Collection<Class<? extends Plugin>> getPlugins() {
+        return Collections.singleton(SpatialPlugin.class);
+    }
+
+    @Override
+    protected void initializeAdditionalMappings(MapperService mapperService) throws IOException {
+        mapperService.merge(docType, new CompressedXContent(Strings.toString(PutMappingRequest.buildFromSimplifiedDef(docType,
+            fieldName(), "type=shape"))), MapperService.MergeReason.MAPPING_UPDATE);
+    }
+
+    protected String fieldName() {
+        return SHAPE_FIELD_NAME;
+    }
+
+    @Override
+    protected ShapeQueryBuilder doCreateTestQueryBuilder() {
+        return doCreateTestQueryBuilder(randomBoolean());
+    }
+
+    protected ShapeQueryBuilder doCreateTestQueryBuilder(boolean indexedShape) {
+        Geometry shape;
+        // multipoint queries not (yet) supported
+        do {
+            shape = ShapeTestUtils.randomGeometry(false);
+        } while (shape.type() == ShapeType.MULTIPOINT || shape.type() == ShapeType.GEOMETRYCOLLECTION);
+
+        ShapeQueryBuilder builder;
+        clearShapeFields();
+        if (indexedShape == false) {
+            builder = new ShapeQueryBuilder(fieldName(), shape);
+        } else {
+            indexedShapeToReturn = shape;
+            indexedShapeId = randomAlphaOfLengthBetween(3, 20);
+            indexedShapeType = randomBoolean() ? randomAlphaOfLengthBetween(3, 20) : null;
+            builder = new ShapeQueryBuilder(fieldName(), indexedShapeId, indexedShapeType);
+            if (randomBoolean()) {
+                indexedShapeIndex = randomAlphaOfLengthBetween(3, 20);
+                builder.indexedShapeIndex(indexedShapeIndex);
+            }
+            if (randomBoolean()) {
+                indexedShapePath = randomAlphaOfLengthBetween(3, 20);
+                builder.indexedShapePath(indexedShapePath);
+            }
+            if (randomBoolean()) {
+                indexedShapeRouting = randomAlphaOfLengthBetween(3, 20);
+                builder.indexedShapeRouting(indexedShapeRouting);
+            }
+        }
+
+        if (shape.type() == ShapeType.LINESTRING || shape.type() == ShapeType.MULTILINESTRING) {
+            builder.relation(randomFrom(ShapeRelation.DISJOINT, ShapeRelation.INTERSECTS));
+        } else {
+            // XYShape does not support CONTAINS:
+            builder.relation(randomFrom(ShapeRelation.DISJOINT, ShapeRelation.INTERSECTS, ShapeRelation.WITHIN));
+        }
+
+        if (randomBoolean()) {
+            builder.ignoreUnmapped(randomBoolean());
+        }
+        return builder;
+    }
+
+    @After
+    public void clearShapeFields() {
+        indexedShapeToReturn = null;
+        indexedShapeId = null;
+        indexedShapeType = null;
+        indexedShapePath = null;
+        indexedShapeIndex = null;
+        indexedShapeRouting = null;
+    }
+
+    @Override
+    protected void doAssertLuceneQuery(ShapeQueryBuilder queryBuilder, Query query, SearchContext context) throws IOException {
+        // Logic for doToQuery is complex and is hard to test here. Need to rely
+        // on Integration tests to determine if created query is correct
+        // TODO improve ShapeQueryBuilder.doToQuery() method to make it
+        // easier to test here
+        assertThat(query, anyOf(instanceOf(BooleanQuery.class), instanceOf(ConstantScoreQuery.class)));
+    }
+
+    public void testNoFieldName() {
+        Geometry shape = ShapeTestUtils.randomGeometry(false);
+        IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () -> new ShapeQueryBuilder(null, shape));
+        assertEquals("fieldName is required", e.getMessage());
+    }
+
+    public void testNoShape() {
+        expectThrows(IllegalArgumentException.class, () -> new ShapeQueryBuilder(fieldName(), (Geometry) null));
+    }
+
+    public void testNoIndexedShape() {
+        IllegalArgumentException e = expectThrows(IllegalArgumentException.class,
+            () -> new ShapeQueryBuilder(fieldName(), null, "type"));
+        assertEquals("either shape or indexedShapeId is required", e.getMessage());
+    }
+
+    public void testNoRelation() {
+        Geometry shape = ShapeTestUtils.randomGeometry(false);
+        ShapeQueryBuilder builder = new ShapeQueryBuilder(fieldName(), shape);
+        IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () -> builder.relation(null));
+        assertEquals("No Shape Relation defined", e.getMessage());
+    }
+
+    public void testFromJson() throws IOException {
+        String json =
+            "{\n" +
+                "  \"shape\" : {\n" +
+                "    \"geometry\" : {\n" +
+                "      \"shape\" : {\n" +
+                "        \"type\" : \"envelope\",\n" +
+                "        \"coordinates\" : [ [ 1300.0, 5300.0 ], [ 1400.0, 5200.0 ] ]\n" +
+                "      },\n" +
+                "      \"relation\" : \"intersects\"\n" +
+                "    },\n" +
+                "    \"ignore_unmapped\" : false,\n" +
+                "    \"boost\" : 42.0\n" +
+                "  }\n" +
+                "}";
+        ShapeQueryBuilder parsed = (ShapeQueryBuilder) parseQuery(json);
+        checkGeneratedJson(json, parsed);
+        assertEquals(json, 42.0, parsed.boost(), 0.0001);
+    }
+
+    @Override
+    public void testMustRewrite() {
+        ShapeQueryBuilder query = doCreateTestQueryBuilder(true);
+
+        UnsupportedOperationException e = expectThrows(UnsupportedOperationException.class, () -> query.toQuery(createShardContext()));
+        assertEquals("query must be rewritten first", e.getMessage());
+        QueryBuilder rewrite = rewriteAndFetch(query, createShardContext());
+        ShapeQueryBuilder geoShapeQueryBuilder = new ShapeQueryBuilder(fieldName(), indexedShapeToReturn);
+        geoShapeQueryBuilder.relation(query.relation());
+        assertEquals(geoShapeQueryBuilder, rewrite);
+    }
+
+    public void testMultipleRewrite() {
+        ShapeQueryBuilder shape = doCreateTestQueryBuilder(true);
+        QueryBuilder builder = new BoolQueryBuilder()
+            .should(shape)
+            .should(shape);
+
+        builder = rewriteAndFetch(builder, createShardContext());
+
+        ShapeQueryBuilder expectedShape = new ShapeQueryBuilder(fieldName(), indexedShapeToReturn);
+        expectedShape.relation(shape.relation());
+        QueryBuilder expected = new BoolQueryBuilder()
+            .should(expectedShape)
+            .should(expectedShape);
+        assertEquals(expected, builder);
+    }
+
+    public void testIgnoreUnmapped() throws IOException {
+        Geometry shape = ShapeTestUtils.randomGeometry(false);
+        final ShapeQueryBuilder queryBuilder = new ShapeQueryBuilder("unmapped", shape);
+        queryBuilder.ignoreUnmapped(true);
+        Query query = queryBuilder.toQuery(createShardContext());
+        assertThat(query, notNullValue());
+        assertThat(query, instanceOf(MatchNoDocsQuery.class));
+
+        final ShapeQueryBuilder failingQueryBuilder = new ShapeQueryBuilder("unmapped", shape);
+        failingQueryBuilder.ignoreUnmapped(false);
+        QueryShardException e = expectThrows(QueryShardException.class, () -> failingQueryBuilder.toQuery(createShardContext()));
+        assertThat(e.getMessage(), containsString("failed to find shape field [unmapped]"));
+    }
+
+    public void testWrongFieldType() {
+        Geometry shape = ShapeTestUtils.randomGeometry(false);
+        final ShapeQueryBuilder queryBuilder = new ShapeQueryBuilder(STRING_FIELD_NAME, shape);
+        QueryShardException e = expectThrows(QueryShardException.class, () -> queryBuilder.toQuery(createShardContext()));
+        assertThat(e.getMessage(), containsString("Field [mapped_string] is not of type [shape] but of type [text]"));
+    }
+
+    public void testSerializationFailsUnlessFetched() throws IOException {
+        QueryBuilder builder = doCreateTestQueryBuilder(true);
+        QueryBuilder queryBuilder = Rewriteable.rewrite(builder, createShardContext());
+        IllegalStateException ise = expectThrows(IllegalStateException.class, () -> queryBuilder.writeTo(new BytesStreamOutput(10)));
+        assertEquals(ise.getMessage(), "supplier must be null, can't serialize suppliers, missing a rewriteAndFetch?");
+        builder = rewriteAndFetch(builder, createShardContext());
+        builder.writeTo(new BytesStreamOutput(10));
+    }
+
+    @Override
+    protected QueryBuilder parseQuery(XContentParser parser) throws IOException {
+        QueryBuilder query = super.parseQuery(parser);
+        assertThat(query, instanceOf(ShapeQueryBuilder.class));
+
+        ShapeQueryBuilder shapeQuery = (ShapeQueryBuilder) query;
+        if (shapeQuery.indexedShapeType() != null) {
+            assertWarnings(ShapeQueryBuilder.TYPES_DEPRECATION_MESSAGE);
+        }
+        return query;
+    }
+
+    @Override
+    protected GetResponse executeGet(GetRequest getRequest) {
+        String indexedType = indexedShapeType != null ? indexedShapeType : MapperService.SINGLE_MAPPING_NAME;
+
+        assertThat(indexedShapeToReturn, notNullValue());
+        assertThat(indexedShapeId, notNullValue());
+        assertThat(getRequest.id(), equalTo(indexedShapeId));
+        assertThat(getRequest.type(), equalTo(indexedType));
+        assertThat(getRequest.routing(), equalTo(indexedShapeRouting));
+        String expectedShapeIndex = indexedShapeIndex == null ? ShapeQueryBuilder.DEFAULT_SHAPE_INDEX_NAME : indexedShapeIndex;
+        assertThat(getRequest.index(), equalTo(expectedShapeIndex));
+        String expectedShapePath = indexedShapePath == null ? ShapeQueryBuilder.DEFAULT_SHAPE_FIELD_NAME : indexedShapePath;
+
+        String json;
+        try {
+            XContentBuilder builder = XContentFactory.jsonBuilder().prettyPrint();
+            builder.startObject();
+            builder.field(expectedShapePath, new ToXContentObject() {
+                @Override
+                public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
+                    return GeoJson.toXContent(indexedShapeToReturn, builder, null);
+                }
+            });
+            builder.field(randomAlphaOfLengthBetween(10, 20), "something");
+            builder.endObject();
+            json = Strings.toString(builder);
+        } catch (IOException ex) {
+            throw new ElasticsearchException("boom", ex);
+        }
+        return new GetResponse(new GetResult(indexedShapeIndex, indexedType, indexedShapeId, 0, 1, 0, true, new BytesArray(json),
+            null, null));
+    }
+}

x-pack/plugin/spatial/src/test/java/org/elasticsearch/xpack/spatial/search/ShapeQueryTests.java

@@ -0,0 +1,236 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License;
+ * you may not use this file except in compliance with the Elastic License.
+ */
+package org.elasticsearch.xpack.spatial.search;
+
+import org.elasticsearch.action.get.GetResponse;
+import org.elasticsearch.action.search.SearchResponse;
+import org.elasticsearch.common.geo.GeoJson;
+import org.elasticsearch.common.geo.ShapeRelation;
+import org.elasticsearch.common.geo.builders.EnvelopeBuilder;
+import org.elasticsearch.common.xcontent.XContentBuilder;
+import org.elasticsearch.common.xcontent.XContentFactory;
+import org.elasticsearch.common.xcontent.XContentType;
+import org.elasticsearch.geo.geometry.Geometry;
+import org.elasticsearch.geo.geometry.ShapeType;
+import org.elasticsearch.index.query.ExistsQueryBuilder;
+import org.elasticsearch.index.query.QueryBuilders;
+import org.elasticsearch.plugins.Plugin;
+import org.elasticsearch.test.ESSingleNodeTestCase;
+import org.elasticsearch.xpack.core.XPackPlugin;
+import org.elasticsearch.xpack.spatial.SpatialPlugin;
+import org.elasticsearch.xpack.spatial.index.query.ShapeQueryBuilder;
+import org.elasticsearch.xpack.spatial.util.ShapeTestUtils;
+import org.locationtech.jts.geom.Coordinate;
+
+import java.util.Collection;
+import java.util.Locale;
+
+import static org.elasticsearch.action.support.WriteRequest.RefreshPolicy.IMMEDIATE;
+import static org.elasticsearch.common.xcontent.XContentFactory.jsonBuilder;
+import static org.elasticsearch.index.query.QueryBuilders.matchAllQuery;
+import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertAcked;
+import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertHitCount;
+import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertSearchResponse;
+import static org.hamcrest.Matchers.containsString;
+import static org.hamcrest.Matchers.equalTo;
+import static org.hamcrest.Matchers.nullValue;
+
+public class ShapeQueryTests extends ESSingleNodeTestCase {
+
+    private static String INDEX = "test";
+    private static String IGNORE_MALFORMED_INDEX = INDEX + "_ignore_malformed";
+    private static String FIELD_TYPE = "geometry";
+    private static String FIELD = "shape";
+    private static Geometry queryGeometry = null;
+
+    private int numDocs;
+
+    @Override
+    public void setUp() throws Exception {
+        super.setUp();
+
+        // create test index
+        assertAcked(client().admin().indices().prepareCreate(INDEX)
+            .addMapping(FIELD_TYPE, FIELD, "type=shape", "alias", "type=alias,path=" + FIELD).get());
+        // create index that ignores malformed geometry
+        assertAcked(client().admin().indices().prepareCreate(IGNORE_MALFORMED_INDEX)
+            .addMapping(FIELD_TYPE, FIELD, "type=shape,ignore_malformed=true", "_source", "enabled=false").get());
+        ensureGreen();
+
+        // index random shapes
+        numDocs = randomIntBetween(25, 50);
+        Geometry geometry;
+        for (int i = 0; i < numDocs; ++i) {
+            geometry = ShapeTestUtils.randomGeometry(false);
+            if (geometry.type() == ShapeType.CIRCLE) continue;
+            if (queryGeometry == null && geometry.type() != ShapeType.MULTIPOINT) {
+                queryGeometry = geometry;
+            }
+            XContentBuilder geoJson = GeoJson.toXContent(geometry, XContentFactory.jsonBuilder()
+                .startObject().field(FIELD), null).endObject();
+
+            try {
+                client().prepareIndex(INDEX, FIELD_TYPE).setSource(geoJson).setRefreshPolicy(IMMEDIATE).get();
+                client().prepareIndex(IGNORE_MALFORMED_INDEX, FIELD_TYPE).setRefreshPolicy(IMMEDIATE).setSource(geoJson).get();
+            } catch (Exception e) {
+                // sometimes GeoTestUtil will create invalid geometry; catch and continue:
+                --i;
+                continue;
+            }
+        }
+    }
+
+    public void testIndexedShapeReferenceSourceDisabled() throws Exception {
+        EnvelopeBuilder shape = new EnvelopeBuilder(new Coordinate(-45, 45), new Coordinate(45, -45));
+
+        client().prepareIndex(IGNORE_MALFORMED_INDEX, FIELD_TYPE, "Big_Rectangle").setSource(jsonBuilder().startObject()
+            .field(FIELD, shape).endObject()).setRefreshPolicy(IMMEDIATE).get();
+
+        IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () -> client().prepareSearch(IGNORE_MALFORMED_INDEX)
+            .setQuery(new ShapeQueryBuilder(FIELD, "Big_Rectangle").indexedShapeIndex(IGNORE_MALFORMED_INDEX)).get());
+        assertThat(e.getMessage(), containsString("source disabled"));
+    }
+
+    public void testShapeFetchingPath() throws Exception {
+        String indexName = "shapes_index";
+        String searchIndex = "search_index";
+        createIndex(indexName);
+        client().admin().indices().prepareCreate(searchIndex).addMapping("type", "location", "type=shape").get();
+
+        String location = "\"location\" : {\"type\":\"polygon\", \"coordinates\":[[[-10,-10],[10,-10],[10,10],[-10,10],[-10,-10]]]}";
+
+        client().prepareIndex(indexName, "type", "1")
+            .setSource(
+                String.format(
+                    Locale.ROOT, "{ %s, \"1\" : { %s, \"2\" : { %s, \"3\" : { %s } }} }", location, location, location, location
+                ), XContentType.JSON)
+            .setRefreshPolicy(IMMEDIATE).get();
+        client().prepareIndex(searchIndex, "type", "1")
+            .setSource(jsonBuilder().startObject().startObject("location")
+                .field("type", "polygon")
+                .startArray("coordinates").startArray()
+                .startArray().value(-20).value(-20).endArray()
+                .startArray().value(20).value(-20).endArray()
+                .startArray().value(20).value(20).endArray()
+                .startArray().value(-20).value(20).endArray()
+                .startArray().value(-20).value(-20).endArray()
+                .endArray().endArray()
+                .endObject().endObject()).setRefreshPolicy(IMMEDIATE).get();
+
+        ShapeQueryBuilder filter = new ShapeQueryBuilder("location", "1").relation(ShapeRelation.INTERSECTS)
+            .indexedShapeIndex(indexName)
+            .indexedShapePath("location");
+        SearchResponse result = client().prepareSearch(searchIndex).setQuery(QueryBuilders.matchAllQuery())
+            .setPostFilter(filter).get();
+        assertSearchResponse(result);
+        assertHitCount(result, 1);
+        filter = new ShapeQueryBuilder("location", "1").relation(ShapeRelation.INTERSECTS)
+            .indexedShapeIndex(indexName)
+            .indexedShapePath("1.location");
+        result = client().prepareSearch(searchIndex).setQuery(QueryBuilders.matchAllQuery())
+            .setPostFilter(filter).get();
+        assertSearchResponse(result);
+        assertHitCount(result, 1);
+        filter = new ShapeQueryBuilder("location", "1").relation(ShapeRelation.INTERSECTS)
+            .indexedShapeIndex(indexName)
+            .indexedShapePath("1.2.location");
+        result = client().prepareSearch(searchIndex).setQuery(QueryBuilders.matchAllQuery())
+            .setPostFilter(filter).get();
+        assertSearchResponse(result);
+        assertHitCount(result, 1);
+        filter = new ShapeQueryBuilder("location", "1").relation(ShapeRelation.INTERSECTS)
+            .indexedShapeIndex(indexName)
+            .indexedShapePath("1.2.3.location");
+        result = client().prepareSearch(searchIndex).setQuery(QueryBuilders.matchAllQuery())
+            .setPostFilter(filter).get();
+        assertSearchResponse(result);
+        assertHitCount(result, 1);
+
+        // now test the query variant
+        ShapeQueryBuilder query = new ShapeQueryBuilder("location", "1")
+            .indexedShapeIndex(indexName)
+            .indexedShapePath("location");
+        result = client().prepareSearch(searchIndex).setQuery(query).get();
+        assertSearchResponse(result);
+        assertHitCount(result, 1);
+        query = new ShapeQueryBuilder("location", "1")
+            .indexedShapeIndex(indexName)
+            .indexedShapePath("1.location");
+        result = client().prepareSearch(searchIndex).setQuery(query).get();
+        assertSearchResponse(result);
+        assertHitCount(result, 1);
+        query = new ShapeQueryBuilder("location", "1")
+            .indexedShapeIndex(indexName)
+            .indexedShapePath("1.2.location");
+        result = client().prepareSearch(searchIndex).setQuery(query).get();
+        assertSearchResponse(result);
+        assertHitCount(result, 1);
+        query = new ShapeQueryBuilder("location", "1")
+            .indexedShapeIndex(indexName)
+            .indexedShapePath("1.2.3.location");
+        result = client().prepareSearch(searchIndex).setQuery(query).get();
+        assertSearchResponse(result);
+        assertHitCount(result, 1);
+    }
+
+    @Override
+    protected Collection<Class<? extends Plugin>> getPlugins() {
+        return pluginList(SpatialPlugin.class, XPackPlugin.class);
+    }
+
+    /**
+     * Test that ignore_malformed on GeoShapeFieldMapper does not fail the entire document
+     */
+    public void testIgnoreMalformed() {
+        assertHitCount(client().prepareSearch(IGNORE_MALFORMED_INDEX).setQuery(matchAllQuery()).get(), numDocs);
+    }
+
+    /**
+     * Test that the indexed shape routing can be provided if it is required
+     */
+    public void testIndexShapeRouting() {
+        String source = "{\n" +
+            "    \"shape\" : {\n" +
+            "        \"type\" : \"bbox\",\n" +
+            "        \"coordinates\" : [[" + -Float.MAX_VALUE + "," +  Float.MAX_VALUE + "], [" + Float.MAX_VALUE + ", " + -Float.MAX_VALUE
+            + "]]\n" +
+            "    }\n" +
+            "}";
+
+        client().prepareIndex(INDEX, FIELD_TYPE, "0").setSource(source, XContentType.JSON).setRouting("ABC").get();
+        client().admin().indices().prepareRefresh(INDEX).get();
+
+        SearchResponse searchResponse = client().prepareSearch(INDEX).setQuery(
+            new ShapeQueryBuilder(FIELD, "0").indexedShapeIndex(INDEX).indexedShapeRouting("ABC")
+        ).get();
+
+        assertThat(searchResponse.getHits().getTotalHits().value, equalTo((long)numDocs+1));
+    }
+
+    public void testNullShape() {
+        // index a null shape
+        client().prepareIndex(INDEX, FIELD_TYPE, "aNullshape").setSource("{\"" + FIELD + "\": null}", XContentType.JSON)
+            .setRefreshPolicy(IMMEDIATE).get();
+        client().prepareIndex(IGNORE_MALFORMED_INDEX, FIELD_TYPE, "aNullshape").setSource("{\"" + FIELD + "\": null}",
+            XContentType.JSON).setRefreshPolicy(IMMEDIATE).get();
+        GetResponse result = client().prepareGet(INDEX, FIELD_TYPE, "aNullshape").get();
+        assertThat(result.getField(FIELD), nullValue());
+    }
+
+    public void testExistsQuery() {
+        ExistsQueryBuilder eqb = QueryBuilders.existsQuery(FIELD);
+        SearchResponse result = client().prepareSearch(INDEX).setQuery(eqb).get();
+        assertSearchResponse(result);
+        assertHitCount(result, numDocs);
+    }
+
+    public void testFieldAlias() {
+        SearchResponse response = client().prepareSearch(INDEX)
+            .setQuery(new ShapeQueryBuilder("alias", queryGeometry).relation(ShapeRelation.INTERSECTS))
+            .get();
+        assertTrue(response.getHits().getTotalHits().value > 0);
+    }
+}

x-pack/plugin/spatial/src/test/java/org/elasticsearch/xpack/spatial/util/ShapeTestUtils.java

@@ -0,0 +1,144 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License;
+ * you may not use this file except in compliance with the Elastic License.
+ */
+package org.elasticsearch.xpack.spatial.util;
+
+import org.apache.lucene.geo.XShapeTestUtil;
+import org.apache.lucene.geo.XYPolygon;
+import org.elasticsearch.geo.GeometryTestUtils;
+import org.elasticsearch.geo.geometry.Geometry;
+import org.elasticsearch.geo.geometry.GeometryCollection;
+import org.elasticsearch.geo.geometry.Line;
+import org.elasticsearch.geo.geometry.LinearRing;
+import org.elasticsearch.geo.geometry.MultiLine;
+import org.elasticsearch.geo.geometry.MultiPoint;
+import org.elasticsearch.geo.geometry.MultiPolygon;
+import org.elasticsearch.geo.geometry.Point;
+import org.elasticsearch.geo.geometry.Polygon;
+import org.elasticsearch.geo.geometry.Rectangle;
+import org.elasticsearch.test.ESTestCase;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.function.Function;
+
+import static org.elasticsearch.geo.GeometryTestUtils.linearRing;
+import static org.elasticsearch.geo.GeometryTestUtils.randomAlt;
+
+/** generates random cartesian shapes */
+public class ShapeTestUtils {
+
+    public static double randomValue() {
+        return XShapeTestUtil.nextDouble();
+    }
+
+    public static Point randomPoint() {
+        return randomPoint(ESTestCase.randomBoolean());
+    }
+
+    public static Point randomPoint(boolean hasAlt) {
+        if (hasAlt) {
+            return new Point(randomValue(), randomValue(), randomAlt());
+        }
+        return new Point(randomValue(), randomValue());
+    }
+
+    public static Line randomLine(boolean hasAlts) {
+        // we use nextPolygon because it guarantees no duplicate points
+        XYPolygon lucenePolygon = XShapeTestUtil.nextPolygon();
+        int size = lucenePolygon.numPoints() - 1;
+        double[] x = new double[size];
+        double[] y = new double[size];
+        double[] alts = hasAlts ? new double[size] : null;
+        for (int i = 0; i < size; i++) {
+            x[i] = lucenePolygon.getPolyX(i);
+            y[i] = lucenePolygon.getPolyY(i);
+            if (hasAlts) {
+                alts[i] = randomAlt();
+            }
+        }
+        if (hasAlts) {
+            return new Line(x, y, alts);
+        }
+        return new Line(x, y);
+    }
+
+    public static Polygon randomPolygon(boolean hasAlt) {
+        XYPolygon lucenePolygon = XShapeTestUtil.nextPolygon();
+        if (lucenePolygon.numHoles() > 0) {
+            XYPolygon[] luceneHoles = lucenePolygon.getHoles();
+            List<LinearRing> holes = new ArrayList<>();
+            for (int i = 0; i < lucenePolygon.numHoles(); i++) {
+                XYPolygon poly = luceneHoles[i];
+                holes.add(linearRing(poly.getPolyY(), poly.getPolyX(), hasAlt));
+            }
+            return new Polygon(linearRing(lucenePolygon.getPolyY(), lucenePolygon.getPolyX(), hasAlt), holes);
+        }
+        return new Polygon(linearRing(lucenePolygon.getPolyY(), lucenePolygon.getPolyX(), hasAlt));
+    }
+
+    public static Rectangle randomRectangle() {
+        org.apache.lucene.geo.XYRectangle rectangle = XShapeTestUtil.nextBox();
+        return new Rectangle(rectangle.minY, rectangle.maxY, rectangle.minX, rectangle.maxX);
+    }
+
+    public static MultiPoint randomMultiPoint(boolean hasAlt) {
+        int size = ESTestCase.randomIntBetween(3, 10);
+        List<Point> points = new ArrayList<>();
+        for (int i = 0; i < size; i++) {
+            points.add(randomPoint(hasAlt));
+        }
+        return new MultiPoint(points);
+    }
+
+    public static MultiLine randomMultiLine(boolean hasAlt) {
+        int size = ESTestCase.randomIntBetween(3, 10);
+        List<Line> lines = new ArrayList<>();
+        for (int i = 0; i < size; i++) {
+            lines.add(randomLine(hasAlt));
+        }
+        return new MultiLine(lines);
+    }
+
+    public static MultiPolygon randomMultiPolygon(boolean hasAlt) {
+        int size = ESTestCase.randomIntBetween(3, 10);
+        List<Polygon> polygons = new ArrayList<>();
+        for (int i = 0; i < size; i++) {
+            polygons.add(randomPolygon(hasAlt));
+        }
+        return new MultiPolygon(polygons);
+    }
+
+    public static GeometryCollection<Geometry> randomGeometryCollection(boolean hasAlt) {
+        return randomGeometryCollection(0, hasAlt);
+    }
+
+    private static GeometryCollection<Geometry> randomGeometryCollection(int level, boolean hasAlt) {
+        int size = ESTestCase.randomIntBetween(1, 10);
+        List<Geometry> shapes = new ArrayList<>();
+        for (int i = 0; i < size; i++) {
+            shapes.add(randomGeometry(level, hasAlt));
+        }
+        return new GeometryCollection<>(shapes);
+    }
+
+    public static Geometry randomGeometry(boolean hasAlt) {
+        return randomGeometry(0, hasAlt);
+    }
+
+    protected static Geometry randomGeometry(int level, boolean hasAlt) {
+        @SuppressWarnings("unchecked") Function<Boolean, Geometry> geometry = ESTestCase.randomFrom(
+            ShapeTestUtils::randomLine,
+            ShapeTestUtils::randomPoint,
+            ShapeTestUtils::randomPolygon,
+            ShapeTestUtils::randomMultiLine,
+            ShapeTestUtils::randomMultiPoint,
+            ShapeTestUtils::randomMultiPolygon,
+            hasAlt ? ShapeTestUtils::randomPoint : (b) -> randomRectangle(),
+            level < 3 ? (b) -> randomGeometryCollection(level + 1, b) : GeometryTestUtils::randomPoint // don't build too deep
+        );
+        return geometry.apply(hasAlt);
+    }
+}