highlighting.asciidoc 30 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917918919920921922923924925926927928929
  1. [[request-body-search-highlighting]]
  2. ==== Highlighting
  3. Highlighters enable you to get highlighted snippets from one or more fields
  4. in your search results so you can show users where the query matches are.
  5. When you request highlights, the response contains an additional `highlight`
  6. element for each search hit that includes the highlighted fields and the
  7. highlighted fragments.
  8. NOTE: Highlighters don't reflect the boolean logic of a query when extracting
  9. terms to highlight. Thus, for some complex boolean queries (e.g nested boolean
  10. queries, queries using `minimum_should_match` etc.), parts of documents may be
  11. highlighted that don't correspond to query matches.
  12. Highlighting requires the actual content of a field. If the field is not
  13. stored (the mapping does not set `store` to `true`), the actual `_source` is
  14. loaded and the relevant field is extracted from `_source`.
  15. For example, to get highlights for the `content` field in each search hit
  16. using the default highlighter, include a `highlight` object in
  17. the request body that specifies the `content` field:
  18. [source,console]
  19. --------------------------------------------------
  20. GET /_search
  21. {
  22. "query" : {
  23. "match": { "content": "kimchy" }
  24. },
  25. "highlight" : {
  26. "fields" : {
  27. "content" : {}
  28. }
  29. }
  30. }
  31. --------------------------------------------------
  32. // TEST[setup:twitter]
  33. {es} supports three highlighters: `unified`, `plain`, and `fvh` (fast vector
  34. highlighter). You can specify the highlighter `type` you want to use
  35. for each field.
  36. [[unified-highlighter]]
  37. ===== Unified highlighter
  38. The `unified` highlighter uses the Lucene Unified Highlighter. This
  39. highlighter breaks the text into sentences and uses the BM25 algorithm to score
  40. individual sentences as if they were documents in the corpus. It also supports
  41. accurate phrase and multi-term (fuzzy, prefix, regex) highlighting. This is the
  42. default highlighter.
  43. [[plain-highlighter]]
  44. ===== Plain highlighter
  45. The `plain` highlighter uses the standard Lucene highlighter. It attempts to
  46. reflect the query matching logic in terms of understanding word importance and
  47. any word positioning criteria in phrase queries.
  48. [WARNING]
  49. The `plain` highlighter works best for highlighting simple query matches in a
  50. single field. To accurately reflect query logic, it creates a tiny in-memory
  51. index and re-runs the original query criteria through Lucene's query execution
  52. planner to get access to low-level match information for the current document.
  53. This is repeated for every field and every document that needs to be highlighted.
  54. If you want to highlight a lot of fields in a lot of documents with complex
  55. queries, we recommend using the `unified` highlighter on `postings` or `term_vector` fields.
  56. [[fast-vector-highlighter]]
  57. ===== Fast vector highlighter
  58. The `fvh` highlighter uses the Lucene Fast Vector highlighter.
  59. This highlighter can be used on fields with `term_vector` set to
  60. `with_positions_offsets` in the mapping. The fast vector highlighter:
  61. * Can be customized with a <<boundary-scanners,`boundary_scanner`>>.
  62. * Requires setting `term_vector` to `with_positions_offsets` which
  63. increases the size of the index
  64. * Can combine matches from multiple fields into one result. See
  65. `matched_fields`
  66. * Can assign different weights to matches at different positions allowing
  67. for things like phrase matches being sorted above term matches when
  68. highlighting a Boosting Query that boosts phrase matches over term matches
  69. [WARNING]
  70. The `fvh` highlighter does not support span queries. If you need support for
  71. span queries, try an alternative highlighter, such as the `unified` highlighter.
  72. [[offsets-strategy]]
  73. ===== Offsets Strategy
  74. To create meaningful search snippets from the terms being queried,
  75. the highlighter needs to know the start and end character offsets of each word
  76. in the original text. These offsets can be obtained from:
  77. * The postings list. If `index_options` is set to `offsets` in the mapping,
  78. the `unified` highlighter uses this information to highlight documents without
  79. re-analyzing the text. It re-runs the original query directly on the postings
  80. and extracts the matching offsets from the index, limiting the collection to
  81. the highlighted documents. This is important if you have large fields because
  82. it doesn't require reanalyzing the text to be highlighted. It also requires less
  83. disk space than using `term_vectors`.
  84. * Term vectors. If `term_vector` information is provided by setting
  85. `term_vector` to `with_positions_offsets` in the mapping, the `unified`
  86. highlighter automatically uses the `term_vector` to highlight the field.
  87. It's fast especially for large fields (> `1MB`) and for highlighting multi-term queries like
  88. `prefix` or `wildcard` because it can access the dictionary of terms for each document.
  89. The `fvh` highlighter always uses term vectors.
  90. * Plain highlighting. This mode is used by the `unified` when there is no other alternative.
  91. It creates a tiny in-memory index and re-runs the original query criteria through
  92. Lucene's query execution planner to get access to low-level match information on
  93. the current document. This is repeated for every field and every document that
  94. needs highlighting. The `plain` highlighter always uses plain highlighting.
  95. [WARNING]
  96. Plain highlighting for large texts may require substantial amount of time and memory.
  97. To protect against this, the maximum number of text characters that will be analyzed has been
  98. limited to 1000000. This default limit can be changed
  99. for a particular index with the index setting `index.highlight.max_analyzed_offset`.
  100. [[highlighting-settings]]
  101. ===== Highlighting Settings
  102. Highlighting settings can be set on a global level and overridden at
  103. the field level.
  104. boundary_chars:: A string that contains each boundary character.
  105. Defaults to `.,!? \t\n`.
  106. boundary_max_scan:: How far to scan for boundary characters. Defaults to `20`.
  107. [[boundary-scanners]]
  108. boundary_scanner:: Specifies how to break the highlighted fragments: `chars`,
  109. `sentence`, or `word`. Only valid for the `unified` and `fvh` highlighters.
  110. Defaults to `sentence` for the `unified` highlighter. Defaults to `chars` for
  111. the `fvh` highlighter.
  112. `chars`::: Use the characters specified by `boundary_chars` as highlighting
  113. boundaries. The `boundary_max_scan` setting controls how far to scan for
  114. boundary characters. Only valid for the `fvh` highlighter.
  115. `sentence`::: Break highlighted fragments at the next sentence boundary, as
  116. determined by Java's
  117. https://docs.oracle.com/javase/8/docs/api/java/text/BreakIterator.html[BreakIterator].
  118. You can specify the locale to use with `boundary_scanner_locale`.
  119. +
  120. NOTE: When used with the `unified` highlighter, the `sentence` scanner splits
  121. sentences bigger than `fragment_size` at the first word boundary next to
  122. `fragment_size`. You can set `fragment_size` to 0 to never split any sentence.
  123. `word`::: Break highlighted fragments at the next word boundary, as determined
  124. by Java's https://docs.oracle.com/javase/8/docs/api/java/text/BreakIterator.html[BreakIterator].
  125. You can specify the locale to use with `boundary_scanner_locale`.
  126. boundary_scanner_locale:: Controls which locale is used to search for sentence
  127. and word boundaries. This parameter takes a form of a language tag,
  128. e.g. `"en-US"`, `"fr-FR"`, `"ja-JP"`. More info can be found in the
  129. https://docs.oracle.com/javase/8/docs/api/java/util/Locale.html#forLanguageTag-java.lang.String-[Locale Language Tag]
  130. documentation. The default value is https://docs.oracle.com/javase/8/docs/api/java/util/Locale.html#ROOT[ Locale.ROOT].
  131. encoder:: Indicates if the snippet should be HTML encoded:
  132. `default` (no encoding) or `html` (HTML-escape the snippet text and then
  133. insert the highlighting tags)
  134. fields:: Specifies the fields to retrieve highlights for. You can use wildcards
  135. to specify fields. For example, you could specify `comment_*` to
  136. get highlights for all <<text,text>> and <<keyword,keyword>> fields
  137. that start with `comment_`.
  138. +
  139. NOTE: Only text and keyword fields are highlighted when you use wildcards.
  140. If you use a custom mapper and want to highlight on a field anyway, you
  141. must explicitly specify that field name.
  142. force_source:: Highlight based on the source even if the field is
  143. stored separately. Defaults to `false`.
  144. fragmenter:: Specifies how text should be broken up in highlight
  145. snippets: `simple` or `span`. Only valid for the `plain` highlighter.
  146. Defaults to `span`.
  147. `simple`::: Breaks up text into same-sized fragments.
  148. `span`::: Breaks up text into same-sized fragments, but tries to avoid
  149. breaking up text between highlighted terms. This is helpful when you're
  150. querying for phrases. Default.
  151. fragment_offset:: Controls the margin from which you want to start
  152. highlighting. Only valid when using the `fvh` highlighter.
  153. fragment_size:: The size of the highlighted fragment in characters. Defaults
  154. to 100.
  155. highlight_query:: Highlight matches for a query other than the search
  156. query. This is especially useful if you use a rescore query because
  157. those are not taken into account by highlighting by default.
  158. +
  159. IMPORTANT: {es} does not validate that `highlight_query` contains
  160. the search query in any way so it is possible to define it so
  161. legitimate query results are not highlighted. Generally, you should
  162. include the search query as part of the `highlight_query`.
  163. matched_fields:: Combine matches on multiple fields to highlight a single field.
  164. This is most intuitive for multifields that analyze the same string in different
  165. ways. All `matched_fields` must have `term_vector` set to
  166. `with_positions_offsets`, but only the field to which
  167. the matches are combined is loaded so only that field benefits from having
  168. `store` set to `yes`. Only valid for the `fvh` highlighter.
  169. no_match_size:: The amount of text you want to return from the beginning
  170. of the field if there are no matching fragments to highlight. Defaults
  171. to 0 (nothing is returned).
  172. number_of_fragments:: The maximum number of fragments to return. If the
  173. number of fragments is set to 0, no fragments are returned. Instead,
  174. the entire field contents are highlighted and returned. This can be
  175. handy when you need to highlight short texts such as a title or
  176. address, but fragmentation is not required. If `number_of_fragments`
  177. is 0, `fragment_size` is ignored. Defaults to 5.
  178. order:: Sorts highlighted fragments by score when set to `score`. By default,
  179. fragments will be output in the order they appear in the field (order: `none`).
  180. Setting this option to `score` will output the most relevant fragments first.
  181. Each highlighter applies its own logic to compute relevancy scores. See
  182. the document <<highlighter-internal-work, How highlighters work internally>>
  183. for more details how different highlighters find the best fragments.
  184. phrase_limit:: Controls the number of matching phrases in a document that are
  185. considered. Prevents the `fvh` highlighter from analyzing too many phrases
  186. and consuming too much memory. When using `matched_fields`, `phrase_limit`
  187. phrases per matched field are considered. Raising the limit increases query
  188. time and consumes more memory. Only supported by the `fvh` highlighter.
  189. Defaults to 256.
  190. pre_tags:: Use in conjunction with `post_tags` to define the HTML tags
  191. to use for the highlighted text. By default, highlighted text is wrapped
  192. in `<em>` and `</em>` tags. Specify as an array of strings.
  193. post_tags:: Use in conjunction with `pre_tags` to define the HTML tags
  194. to use for the highlighted text. By default, highlighted text is wrapped
  195. in `<em>` and `</em>` tags. Specify as an array of strings.
  196. require_field_match:: By default, only fields that contains a query match are
  197. highlighted. Set `require_field_match` to `false` to highlight all fields.
  198. Defaults to `true`.
  199. tags_schema:: Set to `styled` to use the built-in tag schema. The `styled`
  200. schema defines the following `pre_tags` and defines `post_tags` as
  201. `</em>`.
  202. +
  203. [source,html]
  204. --------------------------------------------------
  205. <em class="hlt1">, <em class="hlt2">, <em class="hlt3">,
  206. <em class="hlt4">, <em class="hlt5">, <em class="hlt6">,
  207. <em class="hlt7">, <em class="hlt8">, <em class="hlt9">,
  208. <em class="hlt10">
  209. --------------------------------------------------
  210. [[highlighter-type]]
  211. type:: The highlighter to use: `unified`, `plain`, or `fvh`. Defaults to
  212. `unified`.
  213. [[highlighting-examples]]
  214. ===== Highlighting Examples
  215. * <<override-global-settings, Override global settings>>
  216. * <<specify-highlight-query, Specify a highlight query>>
  217. * <<set-highlighter-type, Set highlighter type>>
  218. * <<configure-tags, Configure highlighting tags>>
  219. * <<highlight-source, Highlight source>>
  220. * <<highlight-all, Highlight all fields>>
  221. * <<matched-fields, Combine matches on multiple fields>>
  222. * <<explicit-field-order, Explicitly order highlighted fields>>
  223. * <<control-highlighted-frags, Control highlighted fragments>>
  224. * <<highlight-postings-list, Highlight using the postings list>>
  225. * <<specify-fragmenter, Specify a fragmenter for the plain highlighter>>
  226. [[override-global-settings]]
  227. [float]
  228. ==== Override global settings
  229. You can specify highlighter settings globally and selectively override them for
  230. individual fields.
  231. [source,console]
  232. --------------------------------------------------
  233. GET /_search
  234. {
  235. "query" : {
  236. "match": { "user": "kimchy" }
  237. },
  238. "highlight" : {
  239. "number_of_fragments" : 3,
  240. "fragment_size" : 150,
  241. "fields" : {
  242. "body" : { "pre_tags" : ["<em>"], "post_tags" : ["</em>"] },
  243. "blog.title" : { "number_of_fragments" : 0 },
  244. "blog.author" : { "number_of_fragments" : 0 },
  245. "blog.comment" : { "number_of_fragments" : 5, "order" : "score" }
  246. }
  247. }
  248. }
  249. --------------------------------------------------
  250. // TEST[setup:twitter]
  251. [float]
  252. [[specify-highlight-query]]
  253. ==== Specify a highlight query
  254. You can specify a `highlight_query` to take additional information into account
  255. when highlighting. For example, the following query includes both the search
  256. query and rescore query in the `highlight_query`. Without the `highlight_query`,
  257. highlighting would only take the search query into account.
  258. [source,console]
  259. --------------------------------------------------
  260. GET /_search
  261. {
  262. "stored_fields": [ "_id" ],
  263. "query" : {
  264. "match": {
  265. "comment": {
  266. "query": "foo bar"
  267. }
  268. }
  269. },
  270. "rescore": {
  271. "window_size": 50,
  272. "query": {
  273. "rescore_query" : {
  274. "match_phrase": {
  275. "comment": {
  276. "query": "foo bar",
  277. "slop": 1
  278. }
  279. }
  280. },
  281. "rescore_query_weight" : 10
  282. }
  283. },
  284. "highlight" : {
  285. "order" : "score",
  286. "fields" : {
  287. "comment" : {
  288. "fragment_size" : 150,
  289. "number_of_fragments" : 3,
  290. "highlight_query": {
  291. "bool": {
  292. "must": {
  293. "match": {
  294. "comment": {
  295. "query": "foo bar"
  296. }
  297. }
  298. },
  299. "should": {
  300. "match_phrase": {
  301. "comment": {
  302. "query": "foo bar",
  303. "slop": 1,
  304. "boost": 10.0
  305. }
  306. }
  307. },
  308. "minimum_should_match": 0
  309. }
  310. }
  311. }
  312. }
  313. }
  314. }
  315. --------------------------------------------------
  316. // TEST[setup:twitter]
  317. [float]
  318. [[set-highlighter-type]]
  319. ==== Set highlighter type
  320. The `type` field allows to force a specific highlighter type.
  321. The allowed values are: `unified`, `plain` and `fvh`.
  322. The following is an example that forces the use of the plain highlighter:
  323. [source,console]
  324. --------------------------------------------------
  325. GET /_search
  326. {
  327. "query" : {
  328. "match": { "user": "kimchy" }
  329. },
  330. "highlight" : {
  331. "fields" : {
  332. "comment" : {"type" : "plain"}
  333. }
  334. }
  335. }
  336. --------------------------------------------------
  337. // TEST[setup:twitter]
  338. [[configure-tags]]
  339. [float]
  340. ==== Configure highlighting tags
  341. By default, the highlighting will wrap highlighted text in `<em>` and
  342. `</em>`. This can be controlled by setting `pre_tags` and `post_tags`,
  343. for example:
  344. [source,console]
  345. --------------------------------------------------
  346. GET /_search
  347. {
  348. "query" : {
  349. "match": { "user": "kimchy" }
  350. },
  351. "highlight" : {
  352. "pre_tags" : ["<tag1>"],
  353. "post_tags" : ["</tag1>"],
  354. "fields" : {
  355. "body" : {}
  356. }
  357. }
  358. }
  359. --------------------------------------------------
  360. // TEST[setup:twitter]
  361. When using the fast vector highlighter, you can specify additional tags and the
  362. "importance" is ordered.
  363. [source,console]
  364. --------------------------------------------------
  365. GET /_search
  366. {
  367. "query" : {
  368. "match": { "user": "kimchy" }
  369. },
  370. "highlight" : {
  371. "pre_tags" : ["<tag1>", "<tag2>"],
  372. "post_tags" : ["</tag1>", "</tag2>"],
  373. "fields" : {
  374. "body" : {}
  375. }
  376. }
  377. }
  378. --------------------------------------------------
  379. // TEST[setup:twitter]
  380. You can also use the built-in `styled` tag schema:
  381. [source,console]
  382. --------------------------------------------------
  383. GET /_search
  384. {
  385. "query" : {
  386. "match": { "user": "kimchy" }
  387. },
  388. "highlight" : {
  389. "tags_schema" : "styled",
  390. "fields" : {
  391. "comment" : {}
  392. }
  393. }
  394. }
  395. --------------------------------------------------
  396. // TEST[setup:twitter]
  397. [float]
  398. [[highlight-source]]
  399. ==== Highlight on source
  400. Forces the highlighting to highlight fields based on the source even if fields
  401. are stored separately. Defaults to `false`.
  402. [source,console]
  403. --------------------------------------------------
  404. GET /_search
  405. {
  406. "query" : {
  407. "match": { "user": "kimchy" }
  408. },
  409. "highlight" : {
  410. "fields" : {
  411. "comment" : {"force_source" : true}
  412. }
  413. }
  414. }
  415. --------------------------------------------------
  416. // TEST[setup:twitter]
  417. [[highlight-all]]
  418. [float]
  419. ==== Highlight in all fields
  420. By default, only fields that contains a query match are highlighted. Set
  421. `require_field_match` to `false` to highlight all fields.
  422. [source,console]
  423. --------------------------------------------------
  424. GET /_search
  425. {
  426. "query" : {
  427. "match": { "user": "kimchy" }
  428. },
  429. "highlight" : {
  430. "require_field_match": false,
  431. "fields": {
  432. "body" : { "pre_tags" : ["<em>"], "post_tags" : ["</em>"] }
  433. }
  434. }
  435. }
  436. --------------------------------------------------
  437. // TEST[setup:twitter]
  438. [[matched-fields]]
  439. [float]
  440. ==== Combine matches on multiple fields
  441. WARNING: This is only supported by the `fvh` highlighter
  442. The Fast Vector Highlighter can combine matches on multiple fields to
  443. highlight a single field. This is most intuitive for multifields that
  444. analyze the same string in different ways. All `matched_fields` must have
  445. `term_vector` set to `with_positions_offsets` but only the field to which
  446. the matches are combined is loaded so only that field would benefit from having
  447. `store` set to `yes`.
  448. In the following examples, `comment` is analyzed by the `english`
  449. analyzer and `comment.plain` is analyzed by the `standard` analyzer.
  450. [source,console]
  451. --------------------------------------------------
  452. GET /_search
  453. {
  454. "query": {
  455. "query_string": {
  456. "query": "comment.plain:running scissors",
  457. "fields": ["comment"]
  458. }
  459. },
  460. "highlight": {
  461. "order": "score",
  462. "fields": {
  463. "comment": {
  464. "matched_fields": ["comment", "comment.plain"],
  465. "type" : "fvh"
  466. }
  467. }
  468. }
  469. }
  470. --------------------------------------------------
  471. // TEST[setup:twitter]
  472. The above matches both "run with scissors" and "running with scissors"
  473. and would highlight "running" and "scissors" but not "run". If both
  474. phrases appear in a large document then "running with scissors" is
  475. sorted above "run with scissors" in the fragments list because there
  476. are more matches in that fragment.
  477. [source,console]
  478. --------------------------------------------------
  479. GET /_search
  480. {
  481. "query": {
  482. "query_string": {
  483. "query": "running scissors",
  484. "fields": ["comment", "comment.plain^10"]
  485. }
  486. },
  487. "highlight": {
  488. "order": "score",
  489. "fields": {
  490. "comment": {
  491. "matched_fields": ["comment", "comment.plain"],
  492. "type" : "fvh"
  493. }
  494. }
  495. }
  496. }
  497. --------------------------------------------------
  498. // TEST[setup:twitter]
  499. The above highlights "run" as well as "running" and "scissors" but
  500. still sorts "running with scissors" above "run with scissors" because
  501. the plain match ("running") is boosted.
  502. [source,console]
  503. --------------------------------------------------
  504. GET /_search
  505. {
  506. "query": {
  507. "query_string": {
  508. "query": "running scissors",
  509. "fields": ["comment", "comment.plain^10"]
  510. }
  511. },
  512. "highlight": {
  513. "order": "score",
  514. "fields": {
  515. "comment": {
  516. "matched_fields": ["comment.plain"],
  517. "type" : "fvh"
  518. }
  519. }
  520. }
  521. }
  522. --------------------------------------------------
  523. // TEST[setup:twitter]
  524. The above query wouldn't highlight "run" or "scissor" but shows that
  525. it is just fine not to list the field to which the matches are combined
  526. (`comment`) in the matched fields.
  527. [NOTE]
  528. Technically it is also fine to add fields to `matched_fields` that
  529. don't share the same underlying string as the field to which the matches
  530. are combined. The results might not make much sense and if one of the
  531. matches is off the end of the text then the whole query will fail.
  532. [NOTE]
  533. ===================================================================
  534. There is a small amount of overhead involved with setting
  535. `matched_fields` to a non-empty array so always prefer
  536. [source,js]
  537. --------------------------------------------------
  538. "highlight": {
  539. "fields": {
  540. "comment": {}
  541. }
  542. }
  543. --------------------------------------------------
  544. // NOTCONSOLE
  545. to
  546. [source,js]
  547. --------------------------------------------------
  548. "highlight": {
  549. "fields": {
  550. "comment": {
  551. "matched_fields": ["comment"],
  552. "type" : "fvh"
  553. }
  554. }
  555. }
  556. --------------------------------------------------
  557. // NOTCONSOLE
  558. ===================================================================
  559. [[explicit-field-order]]
  560. [float]
  561. ==== Explicitly order highlighted fields
  562. Elasticsearch highlights the fields in the order that they are sent, but per the
  563. JSON spec, objects are unordered. If you need to be explicit about the order
  564. in which fields are highlighted specify the `fields` as an array:
  565. [source,console]
  566. --------------------------------------------------
  567. GET /_search
  568. {
  569. "highlight": {
  570. "fields": [
  571. { "title": {} },
  572. { "text": {} }
  573. ]
  574. }
  575. }
  576. --------------------------------------------------
  577. // TEST[setup:twitter]
  578. None of the highlighters built into Elasticsearch care about the order that the
  579. fields are highlighted but a plugin might.
  580. [float]
  581. [[control-highlighted-frags]]
  582. ==== Control highlighted fragments
  583. Each field highlighted can control the size of the highlighted fragment
  584. in characters (defaults to `100`), and the maximum number of fragments
  585. to return (defaults to `5`).
  586. For example:
  587. [source,console]
  588. --------------------------------------------------
  589. GET /_search
  590. {
  591. "query" : {
  592. "match": { "user": "kimchy" }
  593. },
  594. "highlight" : {
  595. "fields" : {
  596. "comment" : {"fragment_size" : 150, "number_of_fragments" : 3}
  597. }
  598. }
  599. }
  600. --------------------------------------------------
  601. // TEST[setup:twitter]
  602. On top of this it is possible to specify that highlighted fragments need
  603. to be sorted by score:
  604. [source,console]
  605. --------------------------------------------------
  606. GET /_search
  607. {
  608. "query" : {
  609. "match": { "user": "kimchy" }
  610. },
  611. "highlight" : {
  612. "order" : "score",
  613. "fields" : {
  614. "comment" : {"fragment_size" : 150, "number_of_fragments" : 3}
  615. }
  616. }
  617. }
  618. --------------------------------------------------
  619. // TEST[setup:twitter]
  620. If the `number_of_fragments` value is set to `0` then no fragments are
  621. produced, instead the whole content of the field is returned, and of
  622. course it is highlighted. This can be very handy if short texts (like
  623. document title or address) need to be highlighted but no fragmentation
  624. is required. Note that `fragment_size` is ignored in this case.
  625. [source,console]
  626. --------------------------------------------------
  627. GET /_search
  628. {
  629. "query" : {
  630. "match": { "user": "kimchy" }
  631. },
  632. "highlight" : {
  633. "fields" : {
  634. "body" : {},
  635. "blog.title" : {"number_of_fragments" : 0}
  636. }
  637. }
  638. }
  639. --------------------------------------------------
  640. // TEST[setup:twitter]
  641. When using `fvh` one can use `fragment_offset`
  642. parameter to control the margin to start highlighting from.
  643. In the case where there is no matching fragment to highlight, the default is
  644. to not return anything. Instead, we can return a snippet of text from the
  645. beginning of the field by setting `no_match_size` (default `0`) to the length
  646. of the text that you want returned. The actual length may be shorter or longer than
  647. specified as it tries to break on a word boundary.
  648. [source,console]
  649. --------------------------------------------------
  650. GET /_search
  651. {
  652. "query" : {
  653. "match": { "user": "kimchy" }
  654. },
  655. "highlight" : {
  656. "fields" : {
  657. "comment" : {
  658. "fragment_size" : 150,
  659. "number_of_fragments" : 3,
  660. "no_match_size": 150
  661. }
  662. }
  663. }
  664. }
  665. --------------------------------------------------
  666. // TEST[setup:twitter]
  667. [float]
  668. [[highlight-postings-list]]
  669. ==== Highlight using the postings list
  670. Here is an example of setting the `comment` field in the index mapping to
  671. allow for highlighting using the postings:
  672. [source,console]
  673. --------------------------------------------------
  674. PUT /example
  675. {
  676. "mappings": {
  677. "properties": {
  678. "comment" : {
  679. "type": "text",
  680. "index_options" : "offsets"
  681. }
  682. }
  683. }
  684. }
  685. --------------------------------------------------
  686. Here is an example of setting the `comment` field to allow for
  687. highlighting using the `term_vectors` (this will cause the index to be bigger):
  688. [source,console]
  689. --------------------------------------------------
  690. PUT /example
  691. {
  692. "mappings": {
  693. "properties": {
  694. "comment" : {
  695. "type": "text",
  696. "term_vector" : "with_positions_offsets"
  697. }
  698. }
  699. }
  700. }
  701. --------------------------------------------------
  702. [float]
  703. [[specify-fragmenter]]
  704. ==== Specify a fragmenter for the plain highlighter
  705. When using the `plain` highlighter, you can choose between the `simple` and
  706. `span` fragmenters:
  707. [source,console]
  708. --------------------------------------------------
  709. GET twitter/_search
  710. {
  711. "query" : {
  712. "match_phrase": { "message": "number 1" }
  713. },
  714. "highlight" : {
  715. "fields" : {
  716. "message" : {
  717. "type": "plain",
  718. "fragment_size" : 15,
  719. "number_of_fragments" : 3,
  720. "fragmenter": "simple"
  721. }
  722. }
  723. }
  724. }
  725. --------------------------------------------------
  726. // TEST[setup:twitter]
  727. Response:
  728. [source,console-result]
  729. --------------------------------------------------
  730. {
  731. ...
  732. "hits": {
  733. "total" : {
  734. "value": 1,
  735. "relation": "eq"
  736. },
  737. "max_score": 1.6011951,
  738. "hits": [
  739. {
  740. "_index": "twitter",
  741. "_id": "1",
  742. "_score": 1.6011951,
  743. "_source": {
  744. "user": "test",
  745. "message": "some message with the number 1",
  746. "date": "2009-11-15T14:12:12",
  747. "likes": 1
  748. },
  749. "highlight": {
  750. "message": [
  751. " with the <em>number</em>",
  752. " <em>1</em>"
  753. ]
  754. }
  755. }
  756. ]
  757. }
  758. }
  759. --------------------------------------------------
  760. // TESTRESPONSE[s/\.\.\./"took": $body.took,"timed_out": false,"_shards": $body._shards,/]
  761. [source,console]
  762. --------------------------------------------------
  763. GET twitter/_search
  764. {
  765. "query" : {
  766. "match_phrase": { "message": "number 1" }
  767. },
  768. "highlight" : {
  769. "fields" : {
  770. "message" : {
  771. "type": "plain",
  772. "fragment_size" : 15,
  773. "number_of_fragments" : 3,
  774. "fragmenter": "span"
  775. }
  776. }
  777. }
  778. }
  779. --------------------------------------------------
  780. // TEST[setup:twitter]
  781. Response:
  782. [source,console-result]
  783. --------------------------------------------------
  784. {
  785. ...
  786. "hits": {
  787. "total" : {
  788. "value": 1,
  789. "relation": "eq"
  790. },
  791. "max_score": 1.6011951,
  792. "hits": [
  793. {
  794. "_index": "twitter",
  795. "_id": "1",
  796. "_score": 1.6011951,
  797. "_source": {
  798. "user": "test",
  799. "message": "some message with the number 1",
  800. "date": "2009-11-15T14:12:12",
  801. "likes": 1
  802. },
  803. "highlight": {
  804. "message": [
  805. " with the <em>number</em> <em>1</em>"
  806. ]
  807. }
  808. }
  809. ]
  810. }
  811. }
  812. --------------------------------------------------
  813. // TESTRESPONSE[s/\.\.\./"took": $body.took,"timed_out": false,"_shards": $body._shards,/]
  814. If the `number_of_fragments` option is set to `0`,
  815. `NullFragmenter` is used which does not fragment the text at all.
  816. This is useful for highlighting the entire contents of a document or field.
  817. include::highlighters-internal.asciidoc[]