SQL: Show/desc commands now support table ids (#33363)

Extend SHOW TABLES, DESCRIBE and SHOW COLUMNS to support table
identifiers not just SQL LIKE pattern.
This allows both Elasticsearch-style multi-index patterns and SQL LIKE.
To disambiguate between the two (as the " vs ' can be easy to miss),
the grammar now requires LIKE keyword as a prefix for all LIKE-like
patterns.

Also added some docs comparing the two types of patterns.

Fix #33294

docs/reference/sql/concepts.asciidoc

@@ -9,7 +9,7 @@ NOTE: This documentation while trying to be complete, does assume the reader has
 
 As a general rule, {es-sql} as the name indicates provides a SQL interface to {es}. As such, it follows the SQL terminology and conventions first, whenever possible. However the backing engine itself is {es} for which {es-sql} was purposely created hence why features or concepts that are not available, or cannot be mapped correctly, in SQL appear
 in {es-sql}.
-Last but not least, {es-sql} tries to obey the https://en.wikipedia.org/wiki/Principle_of_least_astonishment[principle of least suprise], though as all things in the world, everything is relative.
+Last but not least, {es-sql} tries to obey the https://en.wikipedia.org/wiki/Principle_of_least_astonishment[principle of least surprise], though as all things in the world, everything is relative.
 
 === Mapping concepts across SQL and {es}
 
@@ -26,7 +26,7 @@ So let's start from the bottom; these roughly are:
 |`column`
 |`field`
 |In both cases, at the lowest level, data is stored in _named_ entries, of a variety of <<sql-data-types, data types>>, containing _one_ value. SQL calls such an entry a _column_ while {es} a _field_.
-Notice that in {es} a field can contain _multiple_ values of the same type (esentially a list) while in SQL, a _column_ can contain _exactly_ one value of said type.
+Notice that in {es} a field can contain _multiple_ values of the same type (essentially a list) while in SQL, a _column_ can contain _exactly_ one value of said type.
 {es-sql} will do its best to preserve the SQL semantic and, depending on the query, reject those that return fields with more than one value.
 
 |`row`
@@ -43,7 +43,7 @@ Notice that in {es} a field can contain _multiple_ values of the same type (esen
 
 |`catalog` or `database`
 |`cluster` instance 
-|In SQL, `catalog` or `database` are used interchangebly and represent a set of schemas that is, a number of tables.
+|In SQL, `catalog` or `database` are used interchangeably and represent a set of schemas that is, a number of tables.
 In {es} the set of indices available are grouped in a `cluster`. The semantics also differ a bit; a `database` is essentially yet another namespace (which can have some implications on the way data is stored) while an {es} `cluster` is a runtime instance, or rather a set of at least one {es} instance (typically running distributed).
 In practice this means that while in SQL one can potentially have multiple catalogs inside an instance, in {es} one is restricted to only _one_.
 
@@ -62,4 +62,4 @@ Multiple clusters, each with its own namespace, connected to each other in a fed
 
 |===
 
-As one can see while the mapping between the concepts are not exactly one to one and the semantics somewhat different, there are more things in common than differences. In fact, thanks to SQL declarative nature, many concepts can move across {es} transparently and the terminology of the two likely to be used interchangebly through-out the rest of the material.
+As one can see while the mapping between the concepts are not exactly one to one and the semantics somewhat different, there are more things in common than differences. In fact, thanks to SQL declarative nature, many concepts can move across {es} transparently and the terminology of the two likely to be used interchangeably through-out the rest of the material.

docs/reference/sql/language/index-patterns.asciidoc

@@ -0,0 +1,70 @@
+[role="xpack"]
+[testenv="basic"]
+[[sql-index-pattern]]
+== Index patterns
+
+{es-sql} supports two types of patterns for matching multiple indices or tables:
+
+* {es} multi-index
+
+The {es} notation for enumerating, including or excluding <<multi-index,multi index syntax>>
+is supported _as long_ as it is quoted or escaped as a table identifier.
+
+For example:
+
+["source","sql",subs="attributes,callouts,macros"]
+----
+include-tagged::{sql-specs}/docs.csv-spec[showTablesEsMultiIndex]
+----
+
+Notice the pattern is surrounded by double quotes `"`. It enumerated `*` meaning all indices however
+it excludes (due to `-`) all indices that start with `l`.
+This notation is very convenient and powerful as it allows both inclusion and exclusion, depending on
+the target naming convention.
+
+* SQL `LIKE` notation
+
+The common `LIKE` statement (including escaping if needed) to match a wildcard pattern, based on one `_`
+or multiple `%` characters.
+
+Using `SHOW TABLES` command again:
+
+["source","sql",subs="attributes,callouts,macros"]
+----
+include-tagged::{sql-specs}/docs.csv-spec[showTablesLikeWildcard]
+----
+
+The pattern matches all tables that start with `emp`. 
+
+This command supports _escaping_ as well, for example:
+
+["source","sql",subs="attributes,callouts,macros"]
+----
+include-tagged::{sql-specs}/docs.csv-spec[showTablesLikeEscape]
+----
+
+Notice how now `emp%` does not match any tables because `%`, which means match zero or more characters,
+has been escaped by `!` and thus becomes an regular char. And since there is no table named `emp%`,
+an empty table is returned.
+
+In a nutshell, the differences between the two type of patterns are:
+
+[cols="^h,^,^",options="header"]
+|===
+| Feature | Multi index | SQL `LIKE`
+
+| Type of quoting    | `"` | `'`
+| Inclusion          | Yes | Yes
+| Exclusion          | Yes | No
+| Enumeration        | Yes | No
+| One char pattern   | No  | `_`
+| Multi char pattern | `*` | `%`
+| Escaping           | No  | `ESCAPE`
+
+|===
+
+Which one to use, is up to you however try to stick to the same one across your queries for consistency.
+
+NOTE: As the query type of quoting between the two patterns is fairly similar (`"` vs `'`), {es-sql} _always_
+requires the keyword `LIKE` for SQL `LIKE` pattern.
+

docs/reference/sql/language/index.asciidoc

@@ -7,6 +7,8 @@ This chapter describes the SQL semantics supported in X-Pack namely:
 
 <<sql-data-types>>:: Data types
 <<sql-commands>>:: Commands
+<<sql-index-patterns>>:: Index patterns
 
 include::data-types.asciidoc[]
 include::syntax/index.asciidoc[]
+include::index-patterns.asciidoc[]

docs/reference/sql/language/syntax/describe-table.asciidoc

@@ -6,14 +6,14 @@
 .Synopsis
 [source, sql]
 ----
-DESCRIBE table
+DESCRIBE [table identifier<1>|[LIKE pattern<2>]]
 ----
 
 or 
 
 [source, sql]
 ----
-DESC table
+DESC [table identifier<1>|[LIKE pattern<2>]]
 ----
 
 

docs/reference/sql/language/syntax/show-columns.asciidoc

@@ -6,9 +6,15 @@
 .Synopsis
 [source, sql]
 ----
-SHOW COLUMNS [ FROM | IN ] ? table 
+SHOW COLUMNS [ FROM | IN ]? [ table identifier<1> | [ LIKE pattern<2> ] ]
 ----
 
+<1> single table identifier or double quoted es multi index
+<2> SQL LIKE pattern
+
+See <<sql-index-pattern, index patterns>> for more information about
+patterns.
+
 .Description
 
 List the columns in table and their data type (and other attributes).
@@ -17,3 +23,4 @@ List the columns in table and their data type (and other attributes).
 ----
 include-tagged::{sql-specs}/docs.csv-spec[showColumns]
 ----
+

docs/reference/sql/language/syntax/show-functions.asciidoc

@@ -6,7 +6,7 @@
 .Synopsis
 [source, sql]
 ----
-SHOW FUNCTIONS [ LIKE? pattern<1>? ]?
+SHOW FUNCTIONS [ LIKE pattern<1>? ]?
 ----
 
 <1> SQL match pattern

docs/reference/sql/language/syntax/show-tables.asciidoc

@@ -6,10 +6,15 @@
 .Synopsis
 [source, sql]
 ----
-SHOW TABLES [ LIKE? pattern<1>? ]?
+SHOW TABLES [ table identifier<1> | [ LIKE pattern<2> ] ]?
 ----
 
-<1> SQL match pattern
+<1> single table identifier or double quoted es multi index
+<2> SQL LIKE pattern
+
+See <<sql-index-pattern, index patterns>> for more information about
+patterns.
+
 
 .Description
 
@@ -20,7 +25,15 @@ List the tables available to the current user and their type.
 include-tagged::{sql-specs}/docs.csv-spec[showTables]
 ----
 
-The `LIKE` clause can be used to restrict the list of names to the given pattern.
+Match multiple indices by using {es} <<multi-index,multi index syntax>>
+notation:
+
+["source","sql",subs="attributes,callouts,macros"]
+----
+include-tagged::{sql-specs}/docs.csv-spec[showTablesEsMultiIndex]
+----
+
+One can also use the `LIKE` clause to restrict the list of names to the given pattern.
 
 The pattern can be an exact match:
 ["source","sql",subs="attributes,callouts,macros"]

x-pack/plugin/sql/src/main/antlr/SqlBase.g4

@@ -50,18 +50,18 @@ statement
         )*
         ')')?
         statement                                                                                         #debug
-    | SHOW TABLES (LIKE? pattern)?                                                                        #showTables
-    | SHOW COLUMNS (FROM | IN) tableIdentifier                                                            #showColumns
-    | (DESCRIBE | DESC) tableIdentifier                                                                   #showColumns
-    | SHOW FUNCTIONS (LIKE? pattern)?                                                                     #showFunctions
+    | SHOW TABLES (tableLike=likePattern | tableIdent=tableIdentifier)?                                   #showTables
+    | SHOW COLUMNS (FROM | IN) (tableLike=likePattern | tableIdent=tableIdentifier)                       #showColumns
+    | (DESCRIBE | DESC) (tableLike=likePattern | tableIdent=tableIdentifier)                              #showColumns
+    | SHOW FUNCTIONS (likePattern)?                                                                       #showFunctions
     | SHOW SCHEMAS                                                                                        #showSchemas
     | SYS CATALOGS                                                                                        #sysCatalogs
-    | SYS TABLES (CATALOG LIKE? clusterPattern=pattern)?
-                 (LIKE? tablePattern=pattern)?
+    | SYS TABLES (CATALOG clusterLike=likePattern)?
+                 (tableLike=likePattern | tableIdent=tableIdentifier)?
                  (TYPE string (',' string)* )?                                                            #sysTables
     | SYS COLUMNS (CATALOG cluster=string)?
-                  (TABLE LIKE? indexPattern=pattern)?
-                  (LIKE? columnPattern=pattern)?                                                          #sysColumns
+                  (TABLE tableLike=likePattern | tableIdent=tableIdentifier)?
+                  (columnPattern=likePattern)?                                                            #sysColumns
     | SYS TYPES                                                                                           #sysTypes
     | SYS TABLE TYPES                                                                                     #sysTableTypes  
     ;
@@ -189,6 +189,10 @@ predicate
     | IS NOT? kind=NULL
     ;
 
+likePattern
+    : LIKE pattern
+    ;
+    
 pattern
     : value=string patternEscape?
     ;

x-pack/plugin/sql/src/main/java/org/elasticsearch/xpack/sql/analysis/index/IndexResolver.java

@@ -140,14 +140,16 @@ public class IndexResolver {
         boolean retrieveAliases = CollectionUtils.isEmpty(types) || types.contains(IndexType.ALIAS);
         boolean retrieveIndices = CollectionUtils.isEmpty(types) || types.contains(IndexType.INDEX);
 
+        String[] indices = Strings.commaDelimitedListToStringArray(indexWildcard);
         if (retrieveAliases) {
             GetAliasesRequest aliasRequest = new GetAliasesRequest()
                     .local(true)
-                    .aliases(indexWildcard)
+                    .indices(indices)
+                    .aliases(indices)
                     .indicesOptions(IndicesOptions.lenientExpandOpen());
     
             client.admin().indices().getAliases(aliasRequest, ActionListener.wrap(aliases ->
-                            resolveIndices(indexWildcard, javaRegex, aliases, retrieveIndices, listener),
+                            resolveIndices(indices, javaRegex, aliases, retrieveIndices, listener),
                             ex -> {
                                 // with security, two exception can be thrown:
                                 // INFE - if no alias matches
@@ -155,34 +157,36 @@ public class IndexResolver {
     
                                 // in both cases, that is allowed and we continue with the indices request
                                 if (ex instanceof IndexNotFoundException || ex instanceof ElasticsearchSecurityException) {
-                                    resolveIndices(indexWildcard, javaRegex, null, retrieveIndices, listener);
+                                    resolveIndices(indices, javaRegex, null, retrieveIndices, listener);
                                 } else {
                                     listener.onFailure(ex);
                                 }
                             }));
         } else {
-            resolveIndices(indexWildcard, javaRegex, null, retrieveIndices, listener);
+            resolveIndices(indices, javaRegex, null, retrieveIndices, listener);
         }
     }
 
-    private void resolveIndices(String indexWildcard, String javaRegex, GetAliasesResponse aliases,
+    private void resolveIndices(String[] indices, String javaRegex, GetAliasesResponse aliases,
             boolean retrieveIndices, ActionListener<Set<IndexInfo>> listener) {
 
         if (retrieveIndices) {
             GetIndexRequest indexRequest = new GetIndexRequest()
                     .local(true)
-                    .indices(indexWildcard)
+                    .indices(indices)
+                    .features(Feature.SETTINGS)
+                    .includeDefaults(false)
                     .indicesOptions(IndicesOptions.lenientExpandOpen());
     
             client.admin().indices().getIndex(indexRequest,
-                    ActionListener.wrap(indices -> filterResults(indexWildcard, javaRegex, aliases, indices, listener),
+                    ActionListener.wrap(response -> filterResults(javaRegex, aliases, response, listener),
                             listener::onFailure));
         } else {
-            filterResults(indexWildcard, javaRegex, aliases, null, listener);
+            filterResults(javaRegex, aliases, null, listener);
         }
     }
 
-    private void filterResults(String indexWildcard, String javaRegex, GetAliasesResponse aliases, GetIndexResponse indices,
+    private void filterResults(String javaRegex, GetAliasesResponse aliases, GetIndexResponse indices,
             ActionListener<Set<IndexInfo>> listener) {
         
         // since the index name does not support ?, filter the results manually
@@ -302,7 +306,6 @@ public class IndexResolver {
         return new GetIndexRequest()
                 .local(true)
                 .indices(Strings.commaDelimitedListToStringArray(index))
-                .features(Feature.MAPPINGS)
                 //lenient because we throw our own errors looking at the response e.g. if something was not resolved
                 //also because this way security doesn't throw authorization exceptions but rather honours ignore_unavailable
                 .indicesOptions(IndicesOptions.lenientExpandOpen());

x-pack/plugin/sql/src/main/java/org/elasticsearch/xpack/sql/parser/CommandBuilder.java

@@ -121,12 +121,14 @@ abstract class CommandBuilder extends LogicalPlanBuilder {
 
     @Override
     public Object visitShowFunctions(ShowFunctionsContext ctx) {
-        return new ShowFunctions(source(ctx), visitPattern(ctx.pattern()));
+        return new ShowFunctions(source(ctx), visitLikePattern(ctx.likePattern()));
     }
 
     @Override
     public Object visitShowTables(ShowTablesContext ctx) {
-        return new ShowTables(source(ctx), visitPattern(ctx.pattern()));
+        TableIdentifier ti = visitTableIdentifier(ctx.tableIdent);
+        String index = ti != null ? ti.qualifiedIndex() : null;
+        return new ShowTables(source(ctx), index, visitLikePattern(ctx.likePattern()));
     }
 
     @Override
@@ -136,8 +138,9 @@ abstract class CommandBuilder extends LogicalPlanBuilder {
 
     @Override
     public Object visitShowColumns(ShowColumnsContext ctx) {
-        TableIdentifier identifier = visitTableIdentifier(ctx.tableIdentifier());
-        return new ShowColumns(source(ctx), identifier.index());
+        TableIdentifier ti = visitTableIdentifier(ctx.tableIdent);
+        String index = ti != null ? ti.qualifiedIndex() : null;
+        return new ShowColumns(source(ctx), index, visitLikePattern(ctx.likePattern()));
     }
 
     @Override
@@ -172,13 +175,17 @@ abstract class CommandBuilder extends LogicalPlanBuilder {
 
         // if the ODBC enumeration is specified, skip validation
         EnumSet<IndexType> set = types.isEmpty() ? null : EnumSet.copyOf(types);
-        return new SysTables(source(ctx), visitPattern(ctx.clusterPattern), visitPattern(ctx.tablePattern), set, legacyTableType);
+        TableIdentifier ti = visitTableIdentifier(ctx.tableIdent);
+        String index = ti != null ? ti.qualifiedIndex() : null;
+        return new SysTables(source(ctx), visitLikePattern(ctx.clusterLike), index, visitLikePattern(ctx.tableLike), set, legacyTableType);
     }
 
     @Override
     public Object visitSysColumns(SysColumnsContext ctx) {
-        Location loc = source(ctx);
-        return new SysColumns(loc, string(ctx.cluster), visitPattern(ctx.indexPattern), visitPattern(ctx.columnPattern));
+        TableIdentifier ti = visitTableIdentifier(ctx.tableIdent);
+        String index = ti != null ? ti.qualifiedIndex() : null;
+        return new SysColumns(source(ctx), string(ctx.cluster), index, visitLikePattern(ctx.tableLike),
+                visitLikePattern(ctx.columnPattern));
     }
 
     @Override
@@ -190,4 +197,4 @@ abstract class CommandBuilder extends LogicalPlanBuilder {
     public Object visitSysTableTypes(SysTableTypesContext ctx) {
         return new SysTableTypes(source(ctx));
     }
-}
+}

x-pack/plugin/sql/src/main/java/org/elasticsearch/xpack/sql/parser/ExpressionBuilder.java

@@ -63,6 +63,7 @@ import org.elasticsearch.xpack.sql.parser.SqlBaseParser.FunctionExpressionContex
 import org.elasticsearch.xpack.sql.parser.SqlBaseParser.FunctionTemplateContext;
 import org.elasticsearch.xpack.sql.parser.SqlBaseParser.GuidEscapedLiteralContext;
 import org.elasticsearch.xpack.sql.parser.SqlBaseParser.IntegerLiteralContext;
+import org.elasticsearch.xpack.sql.parser.SqlBaseParser.LikePatternContext;
 import org.elasticsearch.xpack.sql.parser.SqlBaseParser.LogicalBinaryContext;
 import org.elasticsearch.xpack.sql.parser.SqlBaseParser.LogicalNotContext;
 import org.elasticsearch.xpack.sql.parser.SqlBaseParser.MatchQueryContext;
@@ -220,6 +221,11 @@ abstract class ExpressionBuilder extends IdentifierBuilder {
         return pCtx.NOT() != null ? new Not(loc, e) : e;
     }
 
+    @Override
+    public LikePattern visitLikePattern(LikePatternContext ctx) {
+        return ctx == null ? null : visitPattern(ctx.pattern());
+    }
+
     @Override
     public LikePattern visitPattern(PatternContext ctx) {
         if (ctx == null) {

x-pack/plugin/sql/src/main/java/org/elasticsearch/xpack/sql/parser/IdentifierBuilder.java

@@ -17,6 +17,10 @@ abstract class IdentifierBuilder extends AbstractBuilder {
 
     @Override
     public TableIdentifier visitTableIdentifier(TableIdentifierContext ctx) {
+        if (ctx == null) {
+            return null;
+        }
+
         Location source = source(ctx);
         ParseTree tree = ctx.name != null ? ctx.name : ctx.TABLE_IDENTIFIER();
         String index = tree.getText();

x-pack/plugin/sql/src/main/java/org/elasticsearch/xpack/sql/parser/SqlBaseBaseListener.java

@@ -551,6 +551,18 @@ class SqlBaseBaseListener implements SqlBaseListener {
    * <p>The default implementation does nothing.</p>
    */
   @Override public void exitPredicate(SqlBaseParser.PredicateContext ctx) { }
+  /**
+   * {@inheritDoc}
+   *
+   * <p>The default implementation does nothing.</p>
+   */
+  @Override public void enterLikePattern(SqlBaseParser.LikePatternContext ctx) { }
+  /**
+   * {@inheritDoc}
+   *
+   * <p>The default implementation does nothing.</p>
+   */
+  @Override public void exitLikePattern(SqlBaseParser.LikePatternContext ctx) { }
   /**
    * {@inheritDoc}
    *

x-pack/plugin/sql/src/main/java/org/elasticsearch/xpack/sql/parser/SqlBaseBaseVisitor.java

@@ -326,6 +326,13 @@ class SqlBaseBaseVisitor<T> extends AbstractParseTreeVisitor<T> implements SqlBa
    * {@link #visitChildren} on {@code ctx}.</p>
    */
   @Override public T visitPredicate(SqlBaseParser.PredicateContext ctx) { return visitChildren(ctx); }
+  /**
+   * {@inheritDoc}
+   *
+   * <p>The default implementation returns the result of calling
+   * {@link #visitChildren} on {@code ctx}.</p>
+   */
+  @Override public T visitLikePattern(SqlBaseParser.LikePatternContext ctx) { return visitChildren(ctx); }
   /**
    * {@inheritDoc}
    *

x-pack/plugin/sql/src/main/java/org/elasticsearch/xpack/sql/parser/SqlBaseListener.java

@@ -509,6 +509,16 @@ interface SqlBaseListener extends ParseTreeListener {
    * @param ctx the parse tree
    */
   void exitPredicate(SqlBaseParser.PredicateContext ctx);
+  /**
+   * Enter a parse tree produced by {@link SqlBaseParser#likePattern}.
+   * @param ctx the parse tree
+   */
+  void enterLikePattern(SqlBaseParser.LikePatternContext ctx);
+  /**
+   * Exit a parse tree produced by {@link SqlBaseParser#likePattern}.
+   * @param ctx the parse tree
+   */
+  void exitLikePattern(SqlBaseParser.LikePatternContext ctx);
   /**
    * Enter a parse tree produced by {@link SqlBaseParser#pattern}.
    * @param ctx the parse tree

x-pack/plugin/sql/src/main/java/org/elasticsearch/xpack/sql/parser/SqlBaseVisitor.java

@@ -306,6 +306,12 @@ interface SqlBaseVisitor<T> extends ParseTreeVisitor<T> {
    * @return the visitor result
    */
   T visitPredicate(SqlBaseParser.PredicateContext ctx);
+  /**
+   * Visit a parse tree produced by {@link SqlBaseParser#likePattern}.
+   * @param ctx the parse tree
+   * @return the visitor result
+   */
+  T visitLikePattern(SqlBaseParser.LikePatternContext ctx);
   /**
    * Visit a parse tree produced by {@link SqlBaseParser#pattern}.
    * @param ctx the parse tree

x-pack/plugin/sql/src/main/java/org/elasticsearch/xpack/sql/plan/TableIdentifier.java

@@ -53,6 +53,10 @@ public class TableIdentifier {
         return location;
     }
 
+    public String qualifiedIndex() {
+        return cluster != null ? cluster + ":" + index : index;
+    }
+
     @Override
     public String toString() {
         StringBuilder builder = new StringBuilder();

x-pack/plugin/sql/src/main/java/org/elasticsearch/xpack/sql/plan/logical/command/ShowColumns.java

@@ -8,6 +8,7 @@ package org.elasticsearch.xpack.sql.plan.logical.command;
 import org.elasticsearch.action.ActionListener;
 import org.elasticsearch.xpack.sql.expression.Attribute;
 import org.elasticsearch.xpack.sql.expression.FieldAttribute;
+import org.elasticsearch.xpack.sql.expression.regex.LikePattern;
 import org.elasticsearch.xpack.sql.session.Rows;
 import org.elasticsearch.xpack.sql.session.SchemaRowSet;
 import org.elasticsearch.xpack.sql.session.SqlSession;
@@ -29,19 +30,25 @@ import static java.util.Collections.emptyList;
 public class ShowColumns extends Command {
 
     private final String index;
+    private final LikePattern pattern;
 
-    public ShowColumns(Location location, String index) {
+    public ShowColumns(Location location, String index, LikePattern pattern) {
         super(location);
         this.index = index;
+        this.pattern = pattern;
     }
 
     public String index() {
         return index;
     }
 
+    public LikePattern pattern() {
+        return pattern;
+    }
+
     @Override
     protected NodeInfo<ShowColumns> info() {
-        return NodeInfo.create(this, ShowColumns::new, index);
+        return NodeInfo.create(this, ShowColumns::new, index, pattern);
     }
 
     @Override
@@ -51,7 +58,9 @@ public class ShowColumns extends Command {
 
     @Override
     public void execute(SqlSession session, ActionListener<SchemaRowSet> listener) {
-        session.indexResolver().resolveWithSameMapping(index, null, ActionListener.wrap(
+        String idx = index != null ? index : (pattern != null ? pattern.asIndexNameWildcard() : "*");
+        String regex = pattern != null ? pattern.asJavaRegex() : null;
+        session.indexResolver().resolveWithSameMapping(idx, regex, ActionListener.wrap(
                 indexResult -> {
                     List<List<?>> rows = emptyList();
                     if (indexResult.isValid()) {
@@ -81,7 +90,7 @@ public class ShowColumns extends Command {
 
     @Override
     public int hashCode() {
-        return Objects.hash(index);
+        return Objects.hash(index, pattern);
     }
 
     @Override
@@ -95,6 +104,7 @@ public class ShowColumns extends Command {
         }
 
         ShowColumns other = (ShowColumns) obj;
-        return Objects.equals(index, other.index);
+        return Objects.equals(index, other.index)
+                && Objects.equals(pattern, other.pattern);
     }
-}
+}

x-pack/plugin/sql/src/main/java/org/elasticsearch/xpack/sql/plan/logical/command/ShowTables.java

@@ -22,16 +22,22 @@ import static java.util.stream.Collectors.toList;
 
 public class ShowTables extends Command {
 
+    private final String index;
     private final LikePattern pattern;
 
-    public ShowTables(Location location, LikePattern pattern) {
+    public ShowTables(Location location, String index, LikePattern pattern) {
         super(location);
+        this.index = index;
         this.pattern = pattern;
     }
 
     @Override
     protected NodeInfo<ShowTables> info() {
-        return NodeInfo.create(this, ShowTables::new, pattern);
+        return NodeInfo.create(this, ShowTables::new, index, pattern);
+    }
+
+    public String index() {
+        return index;
     }
 
     public LikePattern pattern() {
@@ -45,9 +51,9 @@ public class ShowTables extends Command {
 
     @Override
     public final void execute(SqlSession session, ActionListener<SchemaRowSet> listener) {
-        String index = pattern != null ? pattern.asIndexNameWildcard() : "*";
+        String idx = index != null ? index : (pattern != null ? pattern.asIndexNameWildcard() : "*");
         String regex = pattern != null ? pattern.asJavaRegex() : null;
-        session.indexResolver().resolveNames(index, regex, null, ActionListener.wrap(result -> {
+        session.indexResolver().resolveNames(idx, regex, null, ActionListener.wrap(result -> {
             listener.onResponse(Rows.of(output(), result.stream()
                  .map(t -> asList(t.name(), t.type().toSql()))
                 .collect(toList())));
@@ -56,7 +62,7 @@ public class ShowTables extends Command {
 
     @Override
     public int hashCode() {
-        return Objects.hash(pattern);
+        return Objects.hash(index, pattern);
     }
 
     @Override
@@ -70,6 +76,7 @@ public class ShowTables extends Command {
         }
 
         ShowTables other = (ShowTables) obj;
-        return Objects.equals(pattern, other.pattern);
+        return Objects.equals(index, other.index) 
+                && Objects.equals(pattern, other.pattern);
     }
-}
+}

x-pack/plugin/sql/src/main/java/org/elasticsearch/xpack/sql/plan/logical/command/sys/SysColumns.java

@@ -39,19 +39,21 @@ import static org.elasticsearch.xpack.sql.type.DataType.SHORT;
 public class SysColumns extends Command {
 
     private final String catalog;
-    private final LikePattern indexPattern;
+    private final String index;
+    private final LikePattern pattern;
     private final LikePattern columnPattern;
 
-    public SysColumns(Location location, String catalog, LikePattern indexPattern, LikePattern columnPattern) {
+    public SysColumns(Location location, String catalog, String index, LikePattern pattern, LikePattern columnPattern) {
         super(location);
         this.catalog = catalog;
-        this.indexPattern = indexPattern;
+        this.index = index;
+        this.pattern = pattern;
         this.columnPattern = columnPattern;
     }
 
     @Override
     protected NodeInfo<SysColumns> info() {
-        return NodeInfo.create(this, SysColumns::new, catalog, indexPattern, columnPattern);
+        return NodeInfo.create(this, SysColumns::new, catalog, index, pattern, columnPattern);
     }
 
     @Override
@@ -94,12 +96,12 @@ public class SysColumns extends Command {
             return;
         }
 
-        String index = indexPattern != null ? indexPattern.asIndexNameWildcard() : "*";
-        String regex = indexPattern != null ? indexPattern.asJavaRegex() : null;
+        String idx = index != null ? index : (pattern != null ? pattern.asIndexNameWildcard() : "*");
+        String regex = pattern != null ? pattern.asJavaRegex() : null;
 
         Pattern columnMatcher = columnPattern != null ? Pattern.compile(columnPattern.asJavaRegex()) : null;
 
-        session.indexResolver().resolveAsSeparateMappings(index, regex, ActionListener.wrap(esIndices -> {
+        session.indexResolver().resolveAsSeparateMappings(idx, regex, ActionListener.wrap(esIndices -> {
             List<List<?>> rows = new ArrayList<>();
             for (EsIndex esIndex : esIndices) {
                 fillInRows(cluster, esIndex.name(), esIndex.mapping(), null, rows, columnMatcher);
@@ -165,7 +167,7 @@ public class SysColumns extends Command {
     
     @Override
     public int hashCode() {
-        return Objects.hash(catalog, indexPattern, columnPattern);
+        return Objects.hash(catalog, index, pattern, columnPattern);
     }
 
     @Override
@@ -180,7 +182,8 @@ public class SysColumns extends Command {
 
         SysColumns other = (SysColumns) obj;
         return Objects.equals(catalog, other.catalog)
-                && Objects.equals(indexPattern, other.indexPattern)
+                && Objects.equals(index, other.index)
+                && Objects.equals(pattern, other.pattern)
                 && Objects.equals(columnPattern, other.columnPattern);
     }
 }

x-pack/plugin/sql/src/main/java/org/elasticsearch/xpack/sql/plan/logical/command/sys/SysTables.java

@@ -32,16 +32,18 @@ import static org.elasticsearch.xpack.sql.util.StringUtils.SQL_WILDCARD;
 
 public class SysTables extends Command {
 
+    private final String index;
     private final LikePattern pattern;
     private final LikePattern clusterPattern;
     private final EnumSet<IndexType> types;
     // flag indicating whether tables are reported as `TABLE` or `BASE TABLE`
     private final boolean legacyTableTypes;
 
-    public SysTables(Location location, LikePattern clusterPattern, LikePattern pattern, EnumSet<IndexType> types,
+    public SysTables(Location location, LikePattern clusterPattern, String index, LikePattern pattern, EnumSet<IndexType> types,
             boolean legacyTableTypes) {
         super(location);
         this.clusterPattern = clusterPattern;
+        this.index = index;
         this.pattern = pattern;
         this.types = types;
         this.legacyTableTypes = legacyTableTypes;
@@ -49,7 +51,7 @@ public class SysTables extends Command {
 
     @Override
     protected NodeInfo<SysTables> info() {
-        return NodeInfo.create(this, SysTables::new, clusterPattern, pattern, types, legacyTableTypes);
+        return NodeInfo.create(this, SysTables::new, clusterPattern, index, pattern, types, legacyTableTypes);
     }
 
     @Override
@@ -111,10 +113,10 @@ public class SysTables extends Command {
             return;
         }
 
-        String index = pattern != null ? pattern.asIndexNameWildcard() : "*";
+        String idx = index != null ? index : (pattern != null ? pattern.asIndexNameWildcard() : "*");
         String regex = pattern != null ? pattern.asJavaRegex() : null;
 
-        session.indexResolver().resolveNames(index, regex, types, ActionListener.wrap(result -> listener.onResponse(
+        session.indexResolver().resolveNames(idx, regex, types, ActionListener.wrap(result -> listener.onResponse(
                 Rows.of(output(), result.stream()
                  // sort by type (which might be legacy), then by name
                  .sorted(Comparator.<IndexInfo, String> comparing(i -> legacyName(i.type()))
@@ -139,7 +141,7 @@ public class SysTables extends Command {
 
     @Override
     public int hashCode() {
-        return Objects.hash(clusterPattern, pattern, types);
+        return Objects.hash(clusterPattern, index, pattern, types);
     }
 
     @Override
@@ -154,6 +156,7 @@ public class SysTables extends Command {
 
         SysTables other = (SysTables) obj;
         return Objects.equals(clusterPattern, other.clusterPattern)
+                && Objects.equals(index, other.index)
                 && Objects.equals(pattern, other.pattern)
                 && Objects.equals(types, other.types);
     }

x-pack/plugin/sql/src/test/java/org/elasticsearch/xpack/sql/optimizer/OptimizerTests.java

@@ -139,7 +139,7 @@ public class OptimizerTests extends ESTestCase {
     }
 
     public void testPruneSubqueryAliases() {
-        ShowTables s = new ShowTables(EMPTY, null);
+        ShowTables s = new ShowTables(EMPTY, null, null);
         SubQueryAlias plan = new SubQueryAlias(EMPTY, s, "show");
         LogicalPlan result = new PruneSubqueryAliases().apply(plan);
         assertEquals(result, s);

x-pack/qa/sql/src/main/resources/command.csv-spec

@@ -121,7 +121,7 @@ ABS             |SCALAR
 ;
 
 showFunctionsWithLeadingPattern
-SHOW FUNCTIONS '%DAY%';
+SHOW FUNCTIONS LIKE '%DAY%';
 
     name:s      |    type:s
 DAY_OF_MONTH    |SCALAR
@@ -133,15 +133,94 @@ MINUTE_OF_DAY   |SCALAR
 ;
 
 showTables
-SHOW TABLES 'test_emp';
+SHOW TABLES;
+
+       name        |     type      
+test_alias         |ALIAS          
+test_alias_emp     |ALIAS          
+test_emp           |BASE TABLE     
+test_emp_copy      |BASE TABLE     
+test_emp_with_nulls|BASE TABLE     
+;
+
+showTablesSimpleLike
+SHOW TABLES LIKE 'test_emp';
+
+ name:s        | type:s
+test_emp       |BASE TABLE
+;
+
+showTablesMultiLike
+SHOW TABLES LIKE 'test_emp%';
+
+ name:s            | type:s
+test_emp           |BASE TABLE     
+test_emp_copy      |BASE TABLE     
+test_emp_with_nulls|BASE TABLE   
+;
+
+showTablesIdentifier
+SHOW TABLES "test_emp";
 
  name:s        | type:s
 test_emp       |BASE TABLE
 ;
 
+showTablesIdentifierPattern
+SHOW TABLES "test_e*,-test_emp";
+
+ name:s            | type:s
+test_emp_copy      |BASE TABLE
+test_emp_with_nulls|BASE TABLE
+;
+
 // DESCRIBE
 
-describe
+describeSimpleLike
+DESCRIBE LIKE 'test_emp';
+
+       column:s      |   type:s
+birth_date           | TIMESTAMP
+dep                  | STRUCT
+dep.dep_id           | VARCHAR
+dep.dep_name         | VARCHAR
+dep.dep_name.keyword | VARCHAR
+dep.from_date        | TIMESTAMP
+dep.to_date          | TIMESTAMP
+emp_no               | INTEGER
+first_name           | VARCHAR
+first_name.keyword   | VARCHAR
+gender               | VARCHAR
+hire_date            | TIMESTAMP
+languages            | TINYINT
+last_name            | VARCHAR
+last_name.keyword    | VARCHAR
+salary               | INTEGER
+;
+
+describeMultiLike
+DESCRIBE LIKE 'test_emp%';
+
+       column:s      |   type:s
+birth_date           | TIMESTAMP
+dep                  | STRUCT
+dep.dep_id           | VARCHAR
+dep.dep_name         | VARCHAR
+dep.dep_name.keyword | VARCHAR
+dep.from_date        | TIMESTAMP
+dep.to_date          | TIMESTAMP
+emp_no               | INTEGER
+first_name           | VARCHAR
+first_name.keyword   | VARCHAR
+gender               | VARCHAR
+hire_date            | TIMESTAMP
+languages            | TINYINT
+last_name            | VARCHAR
+last_name.keyword    | VARCHAR
+salary               | INTEGER
+;
+
+describeSimpleIdentifier
 DESCRIBE "test_emp";
 
        column:s      |   type:s
@@ -164,8 +243,8 @@ salary               | INTEGER
 ;
 
 
-describeIncludeExclude
-DESCRIBE "test_emp*,-test_alias*";
+describeIncludeExcludeIdentifier
+DESCRIBE "test_emp*,-test_emp_*";
 
 column:s             |   type:s
 birth_date           | TIMESTAMP

x-pack/qa/sql/src/main/resources/docs.csv-spec

@@ -148,6 +148,29 @@ emp            |BASE TABLE
 // end::showTablesLikeMixed
 ;
 
+showTablesLikeEscape
+// tag::showTablesLikeEscape
+SHOW TABLES LIKE 'emp!%' ESCAPE '!';
+
+     name      |     type      
+---------------+---------------
+
+// end::showTablesLikeEscape
+;
+
+
+showTablesEsMultiIndex
+// tag::showTablesEsMultiIndex
+SHOW TABLES "*,-l*";
+
+     name      |     type      
+---------------+---------------
+emp            |BASE TABLE     
+employees      |ALIAS
+          
+// end:showTablesEsMultiIndex
+;
+
 ///////////////////////////////
 //
 // Show Functions
@@ -286,7 +309,7 @@ ABS            |SCALAR
 
 showFunctionsWithPattern
 // tag::showFunctionsWithPattern
-SHOW FUNCTIONS '%DAY%';
+SHOW FUNCTIONS LIKE '%DAY%';
 
      name      |     type      
 ---------------+---------------