Tech debt: Add tests to documentation for query rules, search applications (#98266)

* Add tests for query rules

* More tests

* Fix search app tests

* Fix tests

* Add teardown to tests

* Add tests for list search apps call

* Update test in get search application

* Tweak stack trace

* Make response match in test

---------

Co-authored-by: carlosdelest <carlos.delgado@elastic.co>

docs/reference/query-rules/apis/list-query-rulesets.asciidoc

@@ -35,11 +35,131 @@ Requires the `manage_search_query_rules` privilege.
 
 The following example lists all configured query rulesets:
 
+////
+
+[source,console]
+--------------------------------------------------
+PUT _query_rules/ruleset-1
+{
+    "rules": [
+        {
+            "rule_id": "rule-1",
+            "type": "pinned",
+            "criteria": [
+                {
+                    "type": "exact",
+                    "metadata": "query_string",
+                    "values": [ "puggles" ]
+                }
+            ],
+            "actions": {
+                "ids": ["id1"]
+            }
+        }
+    ]
+}
+
+PUT _query_rules/ruleset-2
+{
+    "rules": [
+        {
+            "rule_id": "rule-1",
+            "type": "pinned",
+            "criteria": [
+                {
+                    "type": "exact",
+                    "metadata": "query_string",
+                    "values": [ "puggles" ]
+                }
+            ],
+            "actions": {
+                "ids": ["id1"]
+            }
+        },
+        {
+            "rule_id": "rule-2",
+            "type": "pinned",
+            "criteria": [
+                {
+                    "type": "fuzzy",
+                    "metadata": "query_string",
+                    "values": [ "pugs" ]
+                }
+            ],
+            "actions": {
+                "ids": ["id2"]
+            }
+        }
+    ]
+}
+
+PUT _query_rules/ruleset-3
+{
+    "rules": [
+        {
+            "rule_id": "rule-1",
+            "type": "pinned",
+            "criteria": [
+                {
+                    "type": "exact",
+                    "metadata": "query_string",
+                    "values": [ "puggles" ]
+                }
+            ],
+            "actions": {
+                "ids": ["id1"]
+            }
+        },
+        {
+            "rule_id": "rule-2",
+            "type": "pinned",
+            "criteria": [
+                {
+                    "type": "fuzzy",
+                    "metadata": "query_string",
+                    "values": [ "pugs" ]
+                }
+            ],
+            "actions": {
+                "ids": ["id2"]
+            }
+        },
+        {
+            "rule_id": "rule-3",
+            "type": "pinned",
+            "criteria": [
+                {
+                    "type": "fuzzy",
+                    "metadata": "query_string",
+                    "values": [ "beagles" ]
+                }
+            ],
+            "actions": {
+                "ids": ["id2"]
+            }
+        }
+    ]
+}
+--------------------------------------------------
+// TESTSETUP
+
+[source,console]
+--------------------------------------------------
+DELETE _query_rules/ruleset-1
+
+DELETE _query_rules/ruleset-2
+
+DELETE _query_rules/ruleset-3
+
+--------------------------------------------------
+// TEARDOWN
+
+////
+
 [source,console]
 ----
 GET _query_rules/
 ----
-// TEST[skip:TBD]
 
 The following example lists the first three query rulesets:
 
@@ -47,44 +167,41 @@ The following example lists the first three query rulesets:
 ----
 GET _query_rules/?from=0&size=3
 ----
-// TEST[skip:TBD]
 
 A sample response:
 
 [source,console-result]
 ----
 {
-    "count": 4,
+    "count": 3,
     "results": [
         {
             "ruleset_id": "ruleset-1",
-            "rule_total_count": 10,
-            "rule_criteria_types_counts: {
-                "exact": 5,
-                "fuzzy": 5
+            "rule_total_count": 1,
+            "rule_criteria_types_counts": {
+                "exact": 1
             }
         },
         {
             "ruleset_id": "ruleset-2",
-            "rule_total_count": 15,
-            "rule_criteria_types_counts: {
-                "exact": 5,
-                "fuzzy": 10,
-                "gt": 4
+            "rule_total_count": 2,
+            "rule_criteria_types_counts": {
+                "exact": 1,
+                "fuzzy": 1
             }
         },
         {
             "ruleset_id": "ruleset-3",
-            "rule_total_count": 5,
-            "rule_criteria_types_counts: {
+            "rule_total_count": 3,
+            "rule_criteria_types_counts": {
                 "exact": 1,
-                "contains": 4
+                "fuzzy": 2
             }
         }
     ]
 }
 ----
-// TEST[skip:TBD]
+// TEST[continued]
 
 [NOTE]
 The counts in `rule_criteria_types_counts` may be larger than the value of `rule_total_count`, because a rule may have multiple criteria.

docs/reference/search-application/apis/delete-search-application.asciidoc

@@ -42,8 +42,31 @@ No Search Application matching `name` could be found.
 
 The following example deletes the Search Application named `my-app`:
 
+////
+[source,console]
+----
+PUT /index1
+
+PUT _application/search_application/my-app
+{
+  "indices": [ "index1" ],
+  "template": {
+    "script": {
+      "source": {
+        "query": {
+          "query_string": {
+            "query": "{{query_string}}"
+          }
+        }
+      }
+    }
+  }
+}
+----
+// TESTSETUP
+////
+
 [source,console]
 ----
 DELETE _application/search_application/my-app/
 ----
-// TEST[skip:TBD]

docs/reference/search-application/apis/get-search-application.asciidoc

@@ -43,18 +43,17 @@ No Search Application matching `name` could be found.
 
 The following example gets the Search Application named `my-app`:
 
+////
+
 [source,console]
-----
-GET _application/search_application/my-app/
-----
-// TEST[skip:TBD]
+--------------------------------------------------
+PUT index1
 
-A sample response:
+PUT index2
 
-[source,console-result]
-----
+PUT _application/search_application/my-app
 {
-    "name": "my-app",
+    "indices": ["index1", "index2"],
     "updated_at_millis": 1682105622204,
     "template": {
       "script": {
@@ -73,4 +72,60 @@ A sample response:
       }
   }
 }
+--------------------------------------------------
+// TESTSETUP
+
+[source,console]
+--------------------------------------------------
+DELETE _application/search_application/my-app
+
+DELETE index1
+
+DELETE index2
+--------------------------------------------------
+// TEARDOWN
+
+////
+
+[source,console]
+----
+GET _application/search_application/my-app/
+----
+
+A sample response:
+
+[source,console-result]
+----
+{
+  "name": "my-app",
+  "updated_at_millis": 1682105622204,
+  "template": {
+    "script": {
+      "source": {
+        "query": {
+          "query_string": {
+            "query": "{{query_string}}",
+            "default_field": "{{default_field}}"
+          }
+        }
+      },
+      "lang": "mustache",
+      "options": {
+        "content_type": "application/json;charset=utf-8"
+      },
+      "params": {
+        "query_string": "*",
+        "default_field": "*"
+      }
+    }
+  }
+}
 ----
+// TESTRESPONSE[s/"updated_at_millis": 1682105622204/"updated_at_millis": $body.$_path/]
+
+[NOTE]
+====
+The indices associated with a search application are not returned with the GET response.
+To view the indices, use the <<indices-get-alias, get alias>> API.
+The alias name will match the search application name, for example `my-app` in the example above.
+====

docs/reference/search-application/apis/list-search-applications.asciidoc

@@ -40,19 +40,67 @@ Requires the `manage_search_application` cluster privilege.
 
 The following example lists all configured search applications:
 
+////
+
+[source,console]
+--------------------------------------------------
+PUT index1
+
+PUT _application/search_application/app-1
+{
+  "indices": [ "index1" ],
+  "template": {
+    "script": {
+      "source": {
+        "query": {
+          "query_string": {
+            "query": "{{query_string}}"
+          }
+        }
+      }
+    }
+  }
+}
+
+PUT _application/search_application/app-2
+{
+  "indices": [ "index1" ],
+  "template": {
+    "script": {
+      "source": {
+        "query": {
+          "query_string": {
+            "query": "{{query_string}}"
+          }
+        }
+      }
+    }
+  }
+}
+--------------------------------------------------
+// TESTSETUP
+
+[source,console]
+--------------------------------------------------
+DELETE _application/search_application/app-1
+
+DELETE _application/search_application/app-2
+--------------------------------------------------
+// TEARDOWN
+
+////
+
 [source,console]
 ----
 GET _application/search_application/
 ----
-// TEST[skip:TBD]
 
-The following example lists the first three search applications whose names match the query string `app`:
+The following example queries the first three search applications whose names start with `app`:
 
 [source,console]
 ----
-GET _application/search_application/?from=0&size=3&q=app
+GET _application/search_application?from=0&size=3&q=app*
 ----
-// TEST[skip:TBD]
 
 A sample response:
 
@@ -62,12 +110,15 @@ A sample response:
   "count": 2,
   "results": [
     {
-      "name": "app-1"
+      "name": "app-1",
+      "updated_at_millis": 1690981129366
     },
     {
-      "name": "app-2"
+      "name": "app-2",
+      "updated_at_millis": 1691501823939
     }
-  ],
-  "updated_at_millis": 1682105622204
+  ]
 }
 ----
+// TESTRESPONSE[s/"updated_at_millis": 1690981129366/"updated_at_millis": $body.$_path/]
+// TESTRESPONSE[s/"updated_at_millis": 1691501823939/"updated_at_millis": $body.$_path/]

docs/reference/search-application/apis/put-search-application.asciidoc

@@ -79,11 +79,16 @@ Search Application `<name>` exists and `create` is `true`.
 [[put-search-application-example]]
 ==== {api-examples-title}
 
-The following example creates a new Search Application called `my-app`:
+The following example creates or updates a new Search Application called `my-app`:
 
+////
 [source,console]
 ----
-PUT _application/search_application/my-app?create
+PUT /index1
+
+PUT /index2
+
+PUT _application/search_application/my-app
 {
   "indices": [ "index1", "index2" ],
   "template": {
@@ -122,54 +127,27 @@ PUT _application/search_application/my-app?create
   }
 }
 ----
-// TEST[skip:TBD]
+// TESTSETUP
 
-When the above `dictionary` parameter is specified, the <<search-application-search, search application search>> API will perform the following parameter validation:
+//////////////////////////
 
-* It accepts only the `query_string` and `default_field` parameters
-* It verifies that `query_string` and `default_field` are both strings
-* It accepts `default_field` only if it takes the values `title` or `description`
-
-If the parameters are not valid, the the <<search-application-search, search application search>> API will return an error.
 [source,console]
-----
-POST _application/search_application/my-app/_search
-{
-  "params": {
-    "default_field": "author",
-    "query_string": "Jane"
-  }
-}
-----
-// TEST[skip:TBD]
+--------------------------------------------------
+DELETE _application/search_application/my-app
 
-In the above example, the value of the `default_field` parameter is not valid, therefore Elasticsearch will return an error:
+DELETE /index1
 
-[source,console-result]
-----
-{
-  "error": {
-    "root_cause": [
-      {
-        "type": "validation_exception",
-        "reason": "Validation Failed: 1: $.default_field: does not have a value in the enumeration [title, description];"
-      }
-    ],
-    "type": "validation_exception",
-    "reason": "Validation Failed: 1: $.default_field: does not have a value in the enumeration [title, description];"
-  },
-  "status": 400
-}
-----
-// TEST[skip:TBD]
+DELETE /index2
+--------------------------------------------------
+// TEARDOWN
 
-The following example creates or updates an existing Search Application called `my-app`:
+////
 
 [source,console]
 ----
 PUT _application/search_application/my-app
 {
-  "indices": [ "index1", "index2", "index3" ],
+  "indices": [ "index1", "index2" ],
   "template": {
     "script": {
       "source": {
@@ -206,4 +184,45 @@ PUT _application/search_application/my-app
   }
 }
 ----
-// TEST[skip:TBD]
+
+When the above `dictionary` parameter is specified, the <<search-application-search, search application search>> API will perform the following parameter validation:
+
+* It accepts only the `query_string` and `default_field` parameters
+* It verifies that `query_string` and `default_field` are both strings
+* It accepts `default_field` only if it takes the values `title` or `description`
+
+If the parameters are not valid, the the <<search-application-search, search application search>> API will return an error.
+[source,console]
+----
+POST _application/search_application/my-app/_search
+{
+  "params": {
+    "default_field": "author",
+    "query_string": "Jane"
+  }
+}
+----
+// TEST[catch:bad_request]
+
+In the above example, the value of the `default_field` parameter is not valid, therefore Elasticsearch will return an error:
+
+[source,JSON]
+----
+{
+  "error": {
+    "root_cause": [
+      {
+        "type": "validation_exception",
+        "reason": 'Validation Failed: 1: $.default_field: does not have a value in the enumeration [title, description];',
+        "stack_trace": ...
+      }
+    ],
+    "type": "validation_exception",
+    "reason": 'Validation Failed: 1: $.default_field: does not have a value in the enumeration [title, description];',
+    "stack_trace": ...
+  },
+  "status": 400
+}
+----
+// TESTRESPONSE[s/"stack_trace": \.\.\./"stack_trace": $body.$_path/]
+// Need to use source,JSON above instead console-result, as $ fields are replaced in console-result snippets

docs/reference/search-application/apis/search-application-render-query.asciidoc

@@ -41,12 +41,63 @@ Search Application `<name>` does not exist.
 The following example renders a query for a search application called `my-app`. In this case, the `from` and `size`
 parameters are not specified, so default values are pulled from the search application template.
 
+////
+[source,console]
+----
+PUT /index1
+
+PUT _application/search_application/my-app
+{
+  "indices": ["index1"],
+  "template": {
+    "script": {
+      "lang": "mustache",
+      "source": """
+      {
+        "query": {
+          "multi_match": {
+            "query": "{{query_string}}",
+            "fields": [{{#text_fields}}"{{name}}^{{boost}}",{{/text_fields}}]
+          }
+        },
+        "explain": "{{explain}}",
+        "from": "{{from}}",
+        "size": "{{size}}"
+      }
+      """,
+      "params": {
+        "query_string": "*",
+        "text_fields": [
+          {"name": "title", "boost": 10},
+          {"name": "description", "boost": 5}
+        ],
+        "explain": false,
+        "from": 0,
+        "size": 10
+      }
+    }
+  }
+}
+----
+// TESTSETUP
+
+[source,console]
+--------------------------------------------------
+DELETE _application/search_application/my-app
+
+DELETE index1
+
+--------------------------------------------------
+// TEARDOWN
+
+////
+
 [source,console]
 ----
 POST _application/search_application/my-app/_render_query
 {
   "params": {
-    "value": "my first query",
+    "query_string": "my first query",
     "text_fields": [
         {
             "name": "title",
@@ -60,26 +111,25 @@ POST _application/search_application/my-app/_render_query
   }
 }
 ----
-// TEST[skip:TBD]
 
 A sample response:
 
 [source,console-result]
 ----
 {
-    "size": 10,
     "from": 0,
+    "size": 10,
     "query": {
         "multi_match": {
             "query": "my first query",
             "fields": [
-                "title^10",
-                "text^1"
+                "text^1.0",
+                "title^10.0"
             ]
         }
-    }
+    },
+    "explain": false
 }
-
 ----
-
+// TEST[continued]
 

docs/reference/search-application/apis/search-application-search.asciidoc

@@ -44,6 +44,57 @@ Search Application `<name>` does not exist.
 
 The following example performs a search against a search application called `my-app`:
 
+////
+[source,console]
+----
+PUT /index1
+
+PUT _application/search_application/my-app
+{
+  "indices": ["index1"],
+  "template": {
+    "script": {
+      "lang": "mustache",
+      "source": """
+      {
+        "query": {
+          "multi_match": {
+            "query": "{{query_string}}",
+            "fields": [{{#text_fields}}"{{name}}^{{boost}}"{{^last}},{{/last}}{{/text_fields}}]
+          }
+        },
+        "explain": "{{explain}}",
+        "from": "{{from}}",
+        "size": "{{size}}"
+      }
+      """,
+      "params": {
+        "query_string": "*",
+        "text_fields": [
+          {"name": "title", "boost": 10, "last": false},
+          {"name": "description", "boost": 5, "last": true}
+        ],
+        "explain": false,
+        "from": 0,
+        "size": 10
+      }
+    }
+  }
+}
+----
+// TESTSETUP
+//////////////////////////
+
+[source,console]
+--------------------------------------------------
+DELETE _application/search_application/my-app
+
+DELETE /index1
+--------------------------------------------------
+// TEARDOWN
+
+////
+
 [source,console]
 ----
 POST _application/search_application/my-app/_search
@@ -65,7 +116,6 @@ POST _application/search_application/my-app/_search
   }
 }
 ----
-// TEST[skip:TBD]
 
 The expected results are search results from the query that was run.
 

docs/reference/search/search-your-data/search-using-query-rules.asciidoc

@@ -108,6 +108,14 @@ The following command will create a query ruleset called `my-ruleset` with two p
 * The first rule will generate a <<query-dsl-pinned-query>> pinning the <<mapping-id-field,`_id`>>s `id1` and `id2` when the `user.query` metadata value is a fuzzy match to either `puggles` or `pugs` _and_ the user's location is in the US.
 * The second rule will generate a <<query-dsl-pinned-query>> pinning the <<mapping-id-field, `_id`>> of `id3` specifically from the `my-index-000001` index and `id4` from the `my-index-000002` index when the `user.query` metadata value matches `beagles`.
 
+////
+[source,console]
+----
+PUT /my-index-000001
+----
+// TESTSETUP
+////
+
 [source,console]
 ----
 PUT /_query_rules/my-ruleset
@@ -161,7 +169,6 @@ PUT /_query_rules/my-ruleset
   ]
 }
 ----
-// TEST[skip:TBD]
 
 The API response returns a results of `created` or `updated` depending on whether this was a new or edited ruleset.
 
@@ -171,7 +178,7 @@ The API response returns a results of `created` or `updated` depending on whethe
   "result": "created"
 }
 ----
-// TEST[skip:TBD]
+// TEST[continued]
 
 You can use the <<get-query-ruleset>> call to retrieve the ruleset you just created,
 the <<list-query-rulesets>> call to retrieve a summary of all query rulesets,
@@ -204,7 +211,7 @@ GET /my-index-000001/_search
   }
 }
 ----
-// TEST[skip:TBD]
+// TEST[continued]
 
 This rule query will match against `rule1` in the defined query ruleset, and will convert the organic query into a pinned query with `id1` and `id2` pinned as the top hits.
 Any other matches from the `user.query` query specified in the organic query will be returned below the pinned results.