update-by-query.asciidoc 8.1 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181
  1. [[java-rest-high-document-update-by-query]]
  2. === Update By Query API
  3. [[java-rest-high-document-update-by-query-request]]
  4. ==== Update By Query Request
  5. A `UpdateByQueryRequest` can be used to update documents in an index.
  6. It requires an existing index (or a set of indices) on which the update is to be performed.
  7. The simplest form of a `UpdateByQueryRequest` looks like follows:
  8. ["source","java",subs="attributes,callouts,macros"]
  9. --------------------------------------------------
  10. include-tagged::{doc-tests}/CRUDDocumentationIT.java[update-by-query-request]
  11. --------------------------------------------------
  12. <1> Creates the `UpdateByQueryRequest` on a set of indices.
  13. By default version conflicts abort the `UpdateByQueryRequest` process but you can just count them by settings it to
  14. `proceed` in the request body
  15. ["source","java",subs="attributes,callouts,macros"]
  16. --------------------------------------------------
  17. include-tagged::{doc-tests}/CRUDDocumentationIT.java[update-by-query-request-conflicts]
  18. --------------------------------------------------
  19. <1> Set `proceed` on version conflict
  20. You can limit the documents by adding a type to the source or by adding a query.
  21. ["source","java",subs="attributes,callouts,macros"]
  22. --------------------------------------------------
  23. include-tagged::{doc-tests}/CRUDDocumentationIT.java[update-by-query-request-typeOrQuery]
  24. --------------------------------------------------
  25. <1> Only copy `doc` type
  26. <2> Only copy documents which have field `user` set to `kimchy`
  27. It’s also possible to limit the number of processed documents by setting size.
  28. ["source","java",subs="attributes,callouts,macros"]
  29. --------------------------------------------------
  30. include-tagged::{doc-tests}/CRUDDocumentationIT.java[update-by-query-request-size]
  31. --------------------------------------------------
  32. <1> Only copy 10 documents
  33. By default `UpdateByQueryRequest` uses batches of 1000. You can change the batch size with `setBatchSize`.
  34. ["source","java",subs="attributes,callouts,macros"]
  35. --------------------------------------------------
  36. include-tagged::{doc-tests}/CRUDDocumentationIT.java[update-by-query-request-scrollSize]
  37. --------------------------------------------------
  38. <1> Use batches of 100 documents
  39. Update by query can also use the ingest feature by specifying a `pipeline`.
  40. ["source","java",subs="attributes,callouts,macros"]
  41. --------------------------------------------------
  42. include-tagged::{doc-tests}/CRUDDocumentationIT.java[update-by-query-request-pipeline]
  43. --------------------------------------------------
  44. <1> set pipeline to `my_pipeline`
  45. `UpdateByQueryRequest` also supports a `script` that modifies the document. The following example illustrates that.
  46. ["source","java",subs="attributes,callouts,macros"]
  47. --------------------------------------------------
  48. include-tagged::{doc-tests}/CRUDDocumentationIT.java[update-by-query-request-script]
  49. --------------------------------------------------
  50. <1> `setScript` to increment the `likes` field on all documents with user `kimchy`.
  51. `UpdateByQueryRequest` also helps in automatically parallelizing using `sliced-scroll` to
  52. slice on `_uid`. Use `setSlices` to specify the number of slices to use.
  53. ["source","java",subs="attributes,callouts,macros"]
  54. --------------------------------------------------
  55. include-tagged::{doc-tests}/CRUDDocumentationIT.java[update-by-query-request-slices]
  56. --------------------------------------------------
  57. <1> set number of slices to use
  58. `UpdateByQueryRequest` uses the `scroll` parameter to control how long it keeps the "search context" alive.
  59. ["source","java",subs="attributes,callouts,macros"]
  60. --------------------------------------------------
  61. include-tagged::{doc-tests}/CRUDDocumentationIT.java[update-by-query-request-scroll]
  62. --------------------------------------------------
  63. <1> set scroll time
  64. If you provide routing then the routing is copied to the scroll query, limiting the process to the shards that match
  65. that routing value.
  66. ["source","java",subs="attributes,callouts,macros"]
  67. --------------------------------------------------
  68. include-tagged::{doc-tests}/CRUDDocumentationIT.java[update-by-query-request-routing]
  69. --------------------------------------------------
  70. <1> set routing
  71. ==== Optional arguments
  72. In addition to the options above the following arguments can optionally be also provided:
  73. ["source","java",subs="attributes,callouts,macros"]
  74. --------------------------------------------------
  75. include-tagged::{doc-tests}/CRUDDocumentationIT.java[update-by-query-request-timeout]
  76. --------------------------------------------------
  77. <1> Timeout to wait for the update by query request to be performed as a `TimeValue`
  78. ["source","java",subs="attributes,callouts,macros"]
  79. --------------------------------------------------
  80. include-tagged::{doc-tests}/CRUDDocumentationIT.java[update-by-query-request-refresh]
  81. --------------------------------------------------
  82. <1> Refresh index after calling update by query
  83. ["source","java",subs="attributes,callouts,macros"]
  84. --------------------------------------------------
  85. include-tagged::{doc-tests}/CRUDDocumentationIT.java[update-by-query-request-indicesOptions]
  86. --------------------------------------------------
  87. <1> Set indices options
  88. [[java-rest-high-document-update-by-query-sync]]
  89. ==== Synchronous Execution
  90. ["source","java",subs="attributes,callouts,macros"]
  91. --------------------------------------------------
  92. include-tagged::{doc-tests}/CRUDDocumentationIT.java[update-by-query-execute]
  93. --------------------------------------------------
  94. [[java-rest-high-document-update-by-query-async]]
  95. ==== Asynchronous Execution
  96. The asynchronous execution of an update by query request requires both the `UpdateByQueryRequest`
  97. instance and an `ActionListener` instance to be passed to the asynchronous
  98. method:
  99. ["source","java",subs="attributes,callouts,macros"]
  100. --------------------------------------------------
  101. include-tagged::{doc-tests}/CRUDDocumentationIT.java[update-by-query-execute-async]
  102. --------------------------------------------------
  103. <1> The `UpdateByQueryRequest` to execute and the `ActionListener` to use when
  104. the execution completes
  105. The asynchronous method does not block and returns immediately. Once it is
  106. completed the `ActionListener` is called back using the `onResponse` method
  107. if the execution successfully completed or using the `onFailure` method if
  108. it failed.
  109. A typical listener for `BulkByScrollResponse` looks like:
  110. ["source","java",subs="attributes,callouts,macros"]
  111. --------------------------------------------------
  112. include-tagged::{doc-tests}/CRUDDocumentationIT.java[update-by-query-execute-listener]
  113. --------------------------------------------------
  114. <1> Called when the execution is successfully completed. The response is
  115. provided as an argument and contains a list of individual results for each
  116. operation that was executed. Note that one or more operations might have
  117. failed while the others have been successfully executed.
  118. <2> Called when the whole `UpdateByQueryRequest` fails. In this case the raised
  119. exception is provided as an argument and no operation has been executed.
  120. [[java-rest-high-document-update-by-query-execute-listener-response]]
  121. ==== Update By Query Response
  122. The returned `BulkByScrollResponse` contains information about the executed operations and
  123. allows to iterate over each result as follows:
  124. ["source","java",subs="attributes,callouts,macros"]
  125. --------------------------------------------------
  126. include-tagged::{doc-tests}/CRUDDocumentationIT.java[update-by-query-response]
  127. --------------------------------------------------
  128. <1> Get total time taken
  129. <2> Check if the request timed out
  130. <3> Get total number of docs processed
  131. <4> Number of docs that were updated
  132. <5> Number of docs that were deleted
  133. <6> Number of batches that were executed
  134. <7> Number of skipped docs
  135. <8> Number of version conflicts
  136. <9> Number of times request had to retry bulk index operations
  137. <10> Number of times request had to retry search operations
  138. <11> The total time this request has throttled itself not including the current throttle time if it is currently sleeping
  139. <12> Remaining delay of any current throttle sleep or 0 if not sleeping
  140. <13> Failures during search phase
  141. <14> Failures during bulk index operation