script-score-query.asciidoc 12 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409
  1. [[query-dsl-script-score-query]]
  2. === Script Score Query
  3. The `script_score` allows you to modify the score of documents that are
  4. retrieved by a query. This can be useful if, for example, a score
  5. function is computationally expensive and it is sufficient to compute
  6. the score on a filtered set of documents.
  7. To use `script_score`, you have to define a query and a script -
  8. a function to be used to compute a new score for each document returned
  9. by the query. For more information on scripting see
  10. <<modules-scripting, scripting documentation>>.
  11. Here is an example of using `script_score` to assign each matched document
  12. a score equal to the number of likes divided by 10:
  13. [source,js]
  14. --------------------------------------------------
  15. GET /_search
  16. {
  17. "query" : {
  18. "script_score" : {
  19. "query" : {
  20. "match": { "message": "elasticsearch" }
  21. },
  22. "script" : {
  23. "source" : "doc['likes'].value / 10 "
  24. }
  25. }
  26. }
  27. }
  28. --------------------------------------------------
  29. // CONSOLE
  30. // TEST[setup:twitter]
  31. NOTE: The values returned from `script_score` cannot be negative. In general,
  32. Lucene requires the scores produced by queries to be non-negative in order to
  33. support certain search optimizations.
  34. ==== Accessing the score of a document within a script
  35. Within a script, you can
  36. {ref}/modules-scripting-fields.html#scripting-score[access]
  37. the `_score` variable which represents the current relevance score of a
  38. document.
  39. ==== Predefined functions within a Painless script
  40. You can use any of the available
  41. <<painless-api-reference, painless functions>> in the painless script.
  42. Besides these functions, there are a number of predefined functions
  43. that can help you with scoring. We suggest you to use them instead of
  44. rewriting equivalent functions of your own, as these functions try
  45. to be the most efficient by using the internal mechanisms.
  46. ===== saturation
  47. `saturation(value,k) = value/(k + value)`
  48. [source,js]
  49. --------------------------------------------------
  50. "script" : {
  51. "source" : "saturation(doc['likes'].value, 1)"
  52. }
  53. --------------------------------------------------
  54. // NOTCONSOLE
  55. ===== sigmoid
  56. `sigmoid(value, k, a) = value^a/ (k^a + value^a)`
  57. [source,js]
  58. --------------------------------------------------
  59. "script" : {
  60. "source" : "sigmoid(doc['likes'].value, 2, 1)"
  61. }
  62. --------------------------------------------------
  63. // NOTCONSOLE
  64. [[vector-functions]]
  65. ===== Functions for vector fields
  66. These functions are used for
  67. for <<dense-vector,`dense_vector`>> and
  68. <<sparse-vector,`sparse_vector`>> fields.
  69. For dense_vector fields, `cosineSimilarity` calculates the measure of
  70. cosine similarity between a given query vector and document vectors.
  71. [source,js]
  72. --------------------------------------------------
  73. {
  74. "query": {
  75. "script_score": {
  76. "query": {
  77. "match_all": {}
  78. },
  79. "script": {
  80. "source": "cosineSimilarity(params.query_vector, doc['my_dense_vector']) + 1.0", <1>
  81. "params": {
  82. "query_vector": [4, 3.4, -0.2] <2>
  83. }
  84. }
  85. }
  86. }
  87. }
  88. --------------------------------------------------
  89. // NOTCONSOLE
  90. <1> The script adds 1.0 to the cosine similarity to prevent the score from being negative.
  91. <2> To take advantage of the script optimizations, provide a query vector as a script parameter.
  92. Similarly, for sparse_vector fields, `cosineSimilaritySparse` calculates cosine similarity
  93. between a given query vector and document vectors.
  94. [source,js]
  95. --------------------------------------------------
  96. {
  97. "query": {
  98. "script_score": {
  99. "query": {
  100. "match_all": {}
  101. },
  102. "script": {
  103. "source": "cosineSimilaritySparse(params.query_vector, doc['my_sparse_vector']) + 1.0",
  104. "params": {
  105. "query_vector": {"2": 0.5, "10" : 111.3, "50": -1.3, "113": 14.8, "4545": 156.0}
  106. }
  107. }
  108. }
  109. }
  110. }
  111. --------------------------------------------------
  112. // NOTCONSOLE
  113. For dense_vector fields, `dotProduct` calculates the measure of
  114. dot product between a given query vector and document vectors.
  115. [source,js]
  116. --------------------------------------------------
  117. {
  118. "query": {
  119. "script_score": {
  120. "query": {
  121. "match_all": {}
  122. },
  123. "script": {
  124. "source": """
  125. double value = dotProduct(params.query_vector, doc['my_vector']);
  126. return sigmoid(1, Math.E, -value); <1>
  127. """,
  128. "params": {
  129. "query_vector": [4, 3.4, -0.2]
  130. }
  131. }
  132. }
  133. }
  134. }
  135. --------------------------------------------------
  136. // NOTCONSOLE
  137. <1> Using the standard sigmoid function prevents scores from being negative.
  138. Similarly, for sparse_vector fields, `dotProductSparse` calculates dot product
  139. between a given query vector and document vectors.
  140. [source,js]
  141. --------------------------------------------------
  142. {
  143. "query": {
  144. "script_score": {
  145. "query": {
  146. "match_all": {}
  147. },
  148. "script": {
  149. "source": """
  150. double value = dotProductSparse(params.query_vector, doc['my_sparse_vector']);
  151. return sigmoid(1, Math.E, -value);
  152. """,
  153. "params": {
  154. "query_vector": {"2": 0.5, "10" : 111.3, "50": -1.3, "113": 14.8, "4545": 156.0}
  155. }
  156. }
  157. }
  158. }
  159. }
  160. --------------------------------------------------
  161. // NOTCONSOLE
  162. NOTE: If a document doesn't have a value for a vector field on which
  163. a vector function is executed, 0 is returned as a result
  164. for this document.
  165. NOTE: If a document's dense vector field has a number of dimensions
  166. different from the query's vector, 0 is used for missing dimensions
  167. in the calculations of vector functions.
  168. [[random-score-function]]
  169. ===== Random score function
  170. `random_score` function generates scores that are uniformly distributed
  171. from 0 up to but not including 1.
  172. `randomScore` function has the following syntax:
  173. `randomScore(<seed>, <fieldName>)`.
  174. It has a required parameter - `seed` as an integer value,
  175. and an optional parameter - `fieldName` as a string value.
  176. [source,js]
  177. --------------------------------------------------
  178. "script" : {
  179. "source" : "randomScore(100, '_seq_no')"
  180. }
  181. --------------------------------------------------
  182. // NOTCONSOLE
  183. If the `fieldName` parameter is omitted, the internal Lucene
  184. document ids will be used as a source of randomness. This is very efficient,
  185. but unfortunately not reproducible since documents might be renumbered
  186. by merges.
  187. [source,js]
  188. --------------------------------------------------
  189. "script" : {
  190. "source" : "randomScore(100)"
  191. }
  192. --------------------------------------------------
  193. // NOTCONSOLE
  194. Note that documents that are within the same shard and have the
  195. same value for field will get the same score, so it is usually desirable
  196. to use a field that has unique values for all documents across a shard.
  197. A good default choice might be to use the `_seq_no`
  198. field, whose only drawback is that scores will change if the document is
  199. updated since update operations also update the value of the `_seq_no` field.
  200. [[decay-functions-numeric-fields]]
  201. ===== Decay functions for numeric fields
  202. You can read more about decay functions
  203. {ref}/query-dsl-function-score-query.html#function-decay[here].
  204. * `double decayNumericLinear(double origin, double scale, double offset, double decay, double docValue)`
  205. * `double decayNumericExp(double origin, double scale, double offset, double decay, double docValue)`
  206. * `double decayNumericGauss(double origin, double scale, double offset, double decay, double docValue)`
  207. [source,js]
  208. --------------------------------------------------
  209. "script" : {
  210. "source" : "decayNumericLinear(params.origin, params.scale, params.offset, params.decay, doc['dval'].value)",
  211. "params": { <1>
  212. "origin": 20,
  213. "scale": 10,
  214. "decay" : 0.5,
  215. "offset" : 0
  216. }
  217. }
  218. --------------------------------------------------
  219. // NOTCONSOLE
  220. <1> Using `params` allows to compile the script only once, even if params change.
  221. ===== Decay functions for geo fields
  222. * `double decayGeoLinear(String originStr, String scaleStr, String offsetStr, double decay, GeoPoint docValue)`
  223. * `double decayGeoExp(String originStr, String scaleStr, String offsetStr, double decay, GeoPoint docValue)`
  224. * `double decayGeoGauss(String originStr, String scaleStr, String offsetStr, double decay, GeoPoint docValue)`
  225. [source,js]
  226. --------------------------------------------------
  227. "script" : {
  228. "source" : "decayGeoExp(params.origin, params.scale, params.offset, params.decay, doc['location'].value)",
  229. "params": {
  230. "origin": "40, -70.12",
  231. "scale": "200km",
  232. "offset": "0km",
  233. "decay" : 0.2
  234. }
  235. }
  236. --------------------------------------------------
  237. // NOTCONSOLE
  238. ===== Decay functions for date fields
  239. * `double decayDateLinear(String originStr, String scaleStr, String offsetStr, double decay, JodaCompatibleZonedDateTime docValueDate)`
  240. * `double decayDateExp(String originStr, String scaleStr, String offsetStr, double decay, JodaCompatibleZonedDateTime docValueDate)`
  241. * `double decayDateGauss(String originStr, String scaleStr, String offsetStr, double decay, JodaCompatibleZonedDateTime docValueDate)`
  242. [source,js]
  243. --------------------------------------------------
  244. "script" : {
  245. "source" : "decayDateGauss(params.origin, params.scale, params.offset, params.decay, doc['date'].value)",
  246. "params": {
  247. "origin": "2008-01-01T01:00:00Z",
  248. "scale": "1h",
  249. "offset" : "0",
  250. "decay" : 0.5
  251. }
  252. }
  253. --------------------------------------------------
  254. // NOTCONSOLE
  255. NOTE: Decay functions on dates are limited to dates in the default format
  256. and default time zone. Also calculations with `now` are not supported.
  257. ==== Faster alternatives
  258. Script Score Query calculates the score for every hit (matching document).
  259. There are faster alternative query types that can efficiently skip
  260. non-competitive hits:
  261. * If you want to boost documents on some static fields, use
  262. <<query-dsl-rank-feature-query, Rank Feature Query>>.
  263. ==== Transition from Function Score Query
  264. We are deprecating <<query-dsl-function-score-query, Function Score>>, and
  265. Script Score Query will be a substitute for it.
  266. Here we describe how Function Score Query's functions can be
  267. equivalently implemented in Script Score Query:
  268. [[script-score]]
  269. ===== `script_score`
  270. What you used in `script_score` of the Function Score query, you
  271. can copy into the Script Score query. No changes here.
  272. [[weight]]
  273. ===== `weight`
  274. `weight` function can be implemented in the Script Score query through
  275. the following script:
  276. [source,js]
  277. --------------------------------------------------
  278. "script" : {
  279. "source" : "params.weight * _score",
  280. "params": {
  281. "weight": 2
  282. }
  283. }
  284. --------------------------------------------------
  285. // NOTCONSOLE
  286. [[random-score]]
  287. ===== `random_score`
  288. Use `randomScore` function
  289. as described in <<random-score-function, random score function>>.
  290. [[field-value-factor]]
  291. ===== `field_value_factor`
  292. `field_value_factor` function can be easily implemented through script:
  293. [source,js]
  294. --------------------------------------------------
  295. "script" : {
  296. "source" : "Math.log10(doc['field'].value * params.factor)",
  297. params" : {
  298. "factor" : 5
  299. }
  300. }
  301. --------------------------------------------------
  302. // NOTCONSOLE
  303. For checking if a document has a missing value, you can use
  304. `doc['field'].size() == 0`. For example, this script will use
  305. a value `1` if a document doesn't have a field `field`:
  306. [source,js]
  307. --------------------------------------------------
  308. "script" : {
  309. "source" : "Math.log10((doc['field'].size() == 0 ? 1 : doc['field'].value()) * params.factor)",
  310. params" : {
  311. "factor" : 5
  312. }
  313. }
  314. --------------------------------------------------
  315. // NOTCONSOLE
  316. This table lists how `field_value_factor` modifiers can be implemented
  317. through a script:
  318. [cols="<,<",options="header",]
  319. |=======================================================================
  320. | Modifier | Implementation in Script Score
  321. | `none` | -
  322. | `log` | `Math.log10(doc['f'].value)`
  323. | `log1p` | `Math.log10(doc['f'].value + 1)`
  324. | `log2p` | `Math.log10(doc['f'].value + 2)`
  325. | `ln` | `Math.log(doc['f'].value)`
  326. | `ln1p` | `Math.log(doc['f'].value + 1)`
  327. | `ln2p` | `Math.log(doc['f'].value + 2)`
  328. | `square` | `Math.pow(doc['f'].value, 2)`
  329. | `sqrt` | `Math.sqrt(doc['f'].value)`
  330. | `reciprocal` | `1.0 / doc['f'].value`
  331. |=======================================================================
  332. [[decay-functions]]
  333. ===== `decay functions`
  334. Script Score query has equivalent <<decay-functions, decay functions>>
  335. that can be used in script.