search-template.asciidoc 16 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706
  1. [[search-template]]
  2. === Search Template
  3. Allows you to use the mustache language to pre render search requests.
  4. [source,console]
  5. ------------------------------------------
  6. GET _search/template
  7. {
  8. "source" : {
  9. "query": { "match" : { "{{my_field}}" : "{{my_value}}" } },
  10. "size" : "{{my_size}}"
  11. },
  12. "params" : {
  13. "my_field" : "message",
  14. "my_value" : "foo",
  15. "my_size" : 5
  16. }
  17. }
  18. ------------------------------------------
  19. // TEST[setup:my_index]
  20. [[search-template-api-request]]
  21. ==== {api-request-title}
  22. `GET _search/template`
  23. [[search-template-api-desc]]
  24. ==== {api-description-title}
  25. The `/_search/template` endpoint allows you to use the mustache language to pre-
  26. render search requests, before they are executed and fill existing templates
  27. with template parameters.
  28. For more information on how Mustache templating and what kind of templating you
  29. can do with it check out the https://mustache.github.io/mustache.5.html[online
  30. documentation of the mustache project].
  31. NOTE: The mustache language is implemented in {es} as a sandboxed scripting
  32. language, hence it obeys settings that may be used to enable or disable scripts
  33. per type and context as described in the
  34. <<allowed-script-types-setting, scripting docs>>.
  35. [[search-template-api-path-params]]
  36. ==== {api-path-parms-title}
  37. include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=index]
  38. [[search-template-api-query-params]]
  39. ==== {api-query-parms-title}
  40. include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=allow-no-indices]
  41. +
  42. Defaults to `true`.
  43. `ccs_minimize_roundtrips`::
  44. (Optional, boolean) If `true`, network round-trips are minimized for
  45. cross-cluster search requests. Defaults to `true`.
  46. include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=expand-wildcards]
  47. `explain`::
  48. (Optional, boolean) If `true`, the response includes additional details about
  49. score computation as part of a hit. Defaults to `false`.
  50. `ignore_throttled`::
  51. (Optional, boolean) If `true`, specified concrete, expanded or aliased indices
  52. are not included in the response when throttled. Defaults to `true`.
  53. include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=index-ignore-unavailable]
  54. include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=preference]
  55. `profile`::
  56. (Optional, boolean) If `true`, the query execution is profiled. Defaults
  57. to `false`.
  58. `rest_total_hits_as_int`::
  59. (Optional, boolean) If `true`, `hits.total` are rendered as an integer in
  60. the response. Defaults to `false`.
  61. include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=routing]
  62. include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=scroll]
  63. include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=search_type]
  64. `typed_keys`::
  65. (Optional, boolean) If `true`, aggregation and suggester names are
  66. prefixed by their respective types in the response. Defaults to `false`.
  67. [[search-template-api-request-body]]
  68. ==== {api-request-body-title}
  69. The API request body must contain the search definition template and its parameters.
  70. [[search-template-api-example]]
  71. ==== {api-response-codes-title}
  72. [[pre-registered-templates]]
  73. ===== Store a search template
  74. You can store a search template using the stored scripts API.
  75. [source,console]
  76. ------------------------------------------
  77. POST _scripts/<templateid>
  78. {
  79. "script": {
  80. "lang": "mustache",
  81. "source": {
  82. "query": {
  83. "match": {
  84. "title": "{{query_string}}"
  85. }
  86. }
  87. }
  88. }
  89. }
  90. ------------------------------------------
  91. // TEST[continued]
  92. //////////////////////////
  93. The API returns the following result if the template has been successfully
  94. created:
  95. [source,console-result]
  96. --------------------------------------------------
  97. {
  98. "acknowledged" : true
  99. }
  100. --------------------------------------------------
  101. //////////////////////////
  102. The template can be retrieved by calling
  103. [source,console]
  104. ------------------------------------------
  105. GET _scripts/<templateid>
  106. ------------------------------------------
  107. // TEST[continued]
  108. The API returns the following result:
  109. [source,console-result]
  110. ------------------------------------------
  111. {
  112. "script" : {
  113. "lang" : "mustache",
  114. "source" : "{\"query\":{\"match\":{\"title\":\"{{query_string}}\"}}}",
  115. "options": {
  116. "content_type" : "application/json; charset=UTF-8"
  117. }
  118. },
  119. "_id": "<templateid>",
  120. "found": true
  121. }
  122. ------------------------------------------
  123. This template can be deleted by calling
  124. [source,console]
  125. ------------------------------------------
  126. DELETE _scripts/<templateid>
  127. ------------------------------------------
  128. // TEST[continued]
  129. [[use-registered-templates]]
  130. ===== Using a stored search template
  131. To use a stored template at search time send the following request:
  132. [source,console]
  133. ------------------------------------------
  134. GET _search/template
  135. {
  136. "id": "<templateid>", <1>
  137. "params": {
  138. "query_string": "search for these words"
  139. }
  140. }
  141. ------------------------------------------
  142. // TEST[catch:missing]
  143. <1> Name of the stored template script.
  144. [[_validating_templates]]
  145. ==== Validating a search template
  146. A template can be rendered in a response with given parameters by using the
  147. following request:
  148. [source,console]
  149. ------------------------------------------
  150. GET _render/template
  151. {
  152. "source": "{ \"query\": { \"terms\": {{#toJson}}statuses{{/toJson}} }}",
  153. "params": {
  154. "statuses" : {
  155. "status": [ "pending", "published" ]
  156. }
  157. }
  158. }
  159. ------------------------------------------
  160. The API returns the rendered template:
  161. [source,console-result]
  162. ------------------------------------------
  163. {
  164. "template_output": {
  165. "query": {
  166. "terms": {
  167. "status": [ <1>
  168. "pending",
  169. "published"
  170. ]
  171. }
  172. }
  173. }
  174. }
  175. ------------------------------------------
  176. <1> `status` array has been populated with values from the `params` object.
  177. Stored templates can also be rendered by calling the following request:
  178. [source,js]
  179. ------------------------------------------
  180. GET _render/template/<template_name>
  181. {
  182. "params": {
  183. "..."
  184. }
  185. }
  186. ------------------------------------------
  187. // NOTCONSOLE
  188. [[search-template-explain-parameter]]
  189. ===== Using the explain parameter
  190. You can use the `explain` parameter when running a template:
  191. [source,console]
  192. ------------------------------------------
  193. GET _search/template
  194. {
  195. "id": "my_template",
  196. "params": {
  197. "status": [ "pending", "published" ]
  198. },
  199. "explain": true
  200. }
  201. ------------------------------------------
  202. // TEST[catch:missing]
  203. [[search-template-profile-parameter]]
  204. ===== Profiling
  205. You can use the `profile` parameter when running a template:
  206. [source,console]
  207. ------------------------------------------
  208. GET _search/template
  209. {
  210. "id": "my_template",
  211. "params": {
  212. "status": [ "pending", "published" ]
  213. },
  214. "profile": true
  215. }
  216. ------------------------------------------
  217. // TEST[catch:missing]
  218. [[search-template-query-string-single]]
  219. ===== Filling in a query string with a single value
  220. [source,console]
  221. ------------------------------------------
  222. GET _search/template
  223. {
  224. "source": {
  225. "query": {
  226. "term": {
  227. "message": "{{query_string}}"
  228. }
  229. }
  230. },
  231. "params": {
  232. "query_string": "search for these words"
  233. }
  234. }
  235. ------------------------------------------
  236. // TEST[setup:my_index]
  237. [[search-template-converting-to-json]]
  238. ===== Converting parameters to JSON
  239. The `{{#toJson}}parameter{{/toJson}}` function can be used to convert parameters
  240. like maps and array to their JSON representation:
  241. [source,console]
  242. ------------------------------------------
  243. GET _search/template
  244. {
  245. "source": "{ \"query\": { \"terms\": {{#toJson}}statuses{{/toJson}} }}",
  246. "params": {
  247. "statuses" : {
  248. "status": [ "pending", "published" ]
  249. }
  250. }
  251. }
  252. ------------------------------------------
  253. which is rendered as:
  254. [source,js]
  255. ------------------------------------------
  256. {
  257. "query": {
  258. "terms": {
  259. "status": [
  260. "pending",
  261. "published"
  262. ]
  263. }
  264. }
  265. }
  266. ------------------------------------------
  267. // NOTCONSOLE
  268. A more complex example substitutes an array of JSON objects:
  269. [source,console]
  270. ------------------------------------------
  271. GET _search/template
  272. {
  273. "source": "{\"query\":{\"bool\":{\"must\": {{#toJson}}clauses{{/toJson}} }}}",
  274. "params": {
  275. "clauses": [
  276. { "term": { "user" : "foo" } },
  277. { "term": { "user" : "bar" } }
  278. ]
  279. }
  280. }
  281. ------------------------------------------
  282. which is rendered as:
  283. [source,js]
  284. ------------------------------------------
  285. {
  286. "query": {
  287. "bool": {
  288. "must": [
  289. {
  290. "term": {
  291. "user": "foo"
  292. }
  293. },
  294. {
  295. "term": {
  296. "user": "bar"
  297. }
  298. }
  299. ]
  300. }
  301. }
  302. }
  303. ------------------------------------------
  304. // NOTCONSOLE
  305. [[search-template-concatenate-array]]
  306. ===== Concatenating array of values
  307. The `{{#join}}array{{/join}}` function can be used to concatenate the
  308. values of an array as a comma delimited string:
  309. [source,console]
  310. ------------------------------------------
  311. GET _search/template
  312. {
  313. "source": {
  314. "query": {
  315. "match": {
  316. "emails": "{{#join}}emails{{/join}}"
  317. }
  318. }
  319. },
  320. "params": {
  321. "emails": [ "username@email.com", "lastname@email.com" ]
  322. }
  323. }
  324. ------------------------------------------
  325. which is rendered as:
  326. [source,js]
  327. ------------------------------------------
  328. {
  329. "query" : {
  330. "match" : {
  331. "emails" : "username@email.com,lastname@email.com"
  332. }
  333. }
  334. }
  335. ------------------------------------------
  336. // NOTCONSOLE
  337. The function also accepts a custom delimiter:
  338. [source,console]
  339. ------------------------------------------
  340. GET _search/template
  341. {
  342. "source": {
  343. "query": {
  344. "range": {
  345. "born": {
  346. "gte" : "{{date.min}}",
  347. "lte" : "{{date.max}}",
  348. "format": "{{#join delimiter='||'}}date.formats{{/join delimiter='||'}}"
  349. }
  350. }
  351. }
  352. },
  353. "params": {
  354. "date": {
  355. "min": "2016",
  356. "max": "31/12/2017",
  357. "formats": ["dd/MM/yyyy", "yyyy"]
  358. }
  359. }
  360. }
  361. ------------------------------------------
  362. which is rendered as:
  363. [source,js]
  364. ------------------------------------------
  365. {
  366. "query": {
  367. "range": {
  368. "born": {
  369. "gte": "2016",
  370. "lte": "31/12/2017",
  371. "format": "dd/MM/yyyy||yyyy"
  372. }
  373. }
  374. }
  375. }
  376. ------------------------------------------
  377. // NOTCONSOLE
  378. [[search-template-default-values]]
  379. ===== Default values
  380. A default value is written as `{{var}}{{^var}}default{{/var}}` for instance:
  381. [source,js]
  382. ------------------------------------------
  383. {
  384. "source": {
  385. "query": {
  386. "range": {
  387. "line_no": {
  388. "gte": "{{start}}",
  389. "lte": "{{end}}{{^end}}20{{/end}}"
  390. }
  391. }
  392. }
  393. },
  394. "params": { ... }
  395. }
  396. ------------------------------------------
  397. // NOTCONSOLE
  398. When `params` is `{ "start": 10, "end": 15 }` this query would be rendered as:
  399. [source,js]
  400. ------------------------------------------
  401. {
  402. "range": {
  403. "line_no": {
  404. "gte": "10",
  405. "lte": "15"
  406. }
  407. }
  408. }
  409. ------------------------------------------
  410. // NOTCONSOLE
  411. But when `params` is `{ "start": 10 }` this query would use the default value
  412. for `end`:
  413. [source,js]
  414. ------------------------------------------
  415. {
  416. "range": {
  417. "line_no": {
  418. "gte": "10",
  419. "lte": "20"
  420. }
  421. }
  422. }
  423. ------------------------------------------
  424. // NOTCONSOLE
  425. [[search-template-conditional-clauses]]
  426. ===== Conditional clauses
  427. Conditional clauses cannot be expressed using the JSON form of the template.
  428. Instead, the template *must* be passed as a string. For instance, let's say
  429. we wanted to run a `match` query on the `line` field, and optionally wanted
  430. to filter by line numbers, where `start` and `end` are optional.
  431. The `params` would look like:
  432. [source,js]
  433. ------------------------------------------
  434. {
  435. "params": {
  436. "text": "words to search for",
  437. "line_no": { <1>
  438. "start": 10,
  439. "end": 20
  440. }
  441. }
  442. }
  443. ------------------------------------------
  444. // NOTCONSOLE
  445. <1> The `line_no`, `start`, and `end` parameters are optional.
  446. When written as a query, the template would include invalid JSON, such as
  447. section markers like `{{#line_no}}`:
  448. [source,js]
  449. ------------------------------------------
  450. {
  451. "query": {
  452. "bool": {
  453. "must": {
  454. "match": {
  455. "line": "{{text}}" <1>
  456. }
  457. },
  458. "filter": {
  459. {{#line_no}} <2>
  460. "range": {
  461. "line_no": {
  462. {{#start}} <3>
  463. "gte": "{{start}}" <4>
  464. {{#end}},{{/end}} <5>
  465. {{/start}}
  466. {{#end}} <6>
  467. "lte": "{{end}}" <7>
  468. {{/end}}
  469. }
  470. }
  471. {{/line_no}}
  472. }
  473. }
  474. }
  475. }
  476. ------------------------------------------
  477. // NOTCONSOLE
  478. <1> Fill in the value of param `text`
  479. <2> Include the `range` filter only if `line_no` is specified
  480. <3> Include the `gte` clause only if `line_no.start` is specified
  481. <4> Fill in the value of param `line_no.start`
  482. <5> Add a comma after the `gte` clause only if `line_no.start`
  483. AND `line_no.end` are specified
  484. <6> Include the `lte` clause only if `line_no.end` is specified
  485. <7> Fill in the value of param `line_no.end`
  486. Because search templates cannot include invalid JSON, you can pass the same
  487. query as a string instead:
  488. [source,js]
  489. --------------------
  490. "source": "{\"query\":{\"bool\":{\"must\":{\"match\":{\"line\":\"{{text}}\"}},\"filter\":{{{#line_no}}\"range\":{\"line_no\":{{{#start}}\"gte\":\"{{start}}\"{{#end}},{{/end}}{{/start}}{{#end}}\"lte\":\"{{end}}\"{{/end}}}}{{/line_no}}}}}}"
  491. --------------------
  492. // NOTCONSOLE
  493. [[search-template-encode-urls]]
  494. ===== Encoding URLs
  495. The `{{#url}}value{{/url}}` function can be used to encode a string value
  496. in a HTML encoding form as defined in by the
  497. https://www.w3.org/TR/html4/[HTML specification].
  498. As an example, it is useful to encode a URL:
  499. [source,console]
  500. ------------------------------------------
  501. GET _render/template
  502. {
  503. "source": {
  504. "query": {
  505. "term": {
  506. "http_access_log": "{{#url}}{{host}}/{{page}}{{/url}}"
  507. }
  508. }
  509. },
  510. "params": {
  511. "host": "https://www.elastic.co/",
  512. "page": "learn"
  513. }
  514. }
  515. ------------------------------------------
  516. The previous query will be rendered as:
  517. [source,console-result]
  518. ------------------------------------------
  519. {
  520. "template_output": {
  521. "query": {
  522. "term": {
  523. "http_access_log": "https%3A%2F%2Fwww.elastic.co%2F%2Flearn"
  524. }
  525. }
  526. }
  527. }
  528. ------------------------------------------
  529. [[multi-search-template]]
  530. === Multi Search Template
  531. Allows to execute several search template requests.
  532. [[multi-search-template-api-request]]
  533. ==== {api-request-title}
  534. `GET _msearch/template`
  535. [[multi-search-template-api-desc]]
  536. ==== {api-description-title}
  537. Allows to execute several search template requests within the same API using the
  538. `_msearch/template` endpoint.
  539. The format of the request is similar to the <<search-multi-search, Multi
  540. Search API>> format:
  541. [source,js]
  542. --------------------------------------------------
  543. header\n
  544. body\n
  545. header\n
  546. body\n
  547. --------------------------------------------------
  548. // NOTCONSOLE
  549. The header part supports the same `index`, `search_type`, `preference`, and
  550. `routing` options as the Multi Search API.
  551. The body includes a search template body request and supports inline, stored and
  552. file templates.
  553. [[multi-search-template-api-example]]
  554. ==== {api-examples-title}
  555. [source,js]
  556. --------------------------------------------------
  557. $ cat requests
  558. {"index": "test"}
  559. {"source": {"query": {"match": {"user" : "{{username}}" }}}, "params": {"username": "john"}} <1>
  560. {"source": {"query": {"{{query_type}}": {"name": "{{name}}" }}}, "params": {"query_type": "match_phrase_prefix", "name": "Smith"}}
  561. {"index": "_all"}
  562. {"id": "template_1", "params": {"query_string": "search for these words" }} <2>
  563. $ curl -H "Content-Type: application/x-ndjson" -XGET localhost:9200/_msearch/template --data-binary "@requests"; echo
  564. --------------------------------------------------
  565. // NOTCONSOLE
  566. // Not converting to console because this shows how curl works
  567. <1> Inline search template request
  568. <2> Search template request based on a stored template
  569. The response returns a `responses` array, which includes the search template
  570. response for each search template request matching its order in the original
  571. multi search template request. If there was a complete failure for that specific
  572. search template request, an object with `error` message will be returned in
  573. place of the actual search response.