put_data_frame.asciidoc 3.5 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108
  1. --
  2. :api: put-data-frame-transform
  3. :request: PutDataFrameTransformRequest
  4. :response: AcknowledgedResponse
  5. --
  6. [id="{upid}-{api}"]
  7. === Put Data Frame Transform API
  8. The Put Data Frame Transform API is used to create a new {dataframe-transform}.
  9. The API accepts a +{request}+ object as a request and returns a +{response}+.
  10. [id="{upid}-{api}-request"]
  11. ==== Put Data Frame Request
  12. A +{request}+ requires the following argument:
  13. ["source","java",subs="attributes,callouts,macros"]
  14. --------------------------------------------------
  15. include-tagged::{doc-tests-file}[{api}-request]
  16. --------------------------------------------------
  17. <1> The configuration of the {dataframe-job} to create
  18. [id="{upid}-{api}-config"]
  19. ==== Data Frame Transform Configuration
  20. The `DataFrameTransformConfig` object contains all the details about the {dataframe-transform}
  21. configuration and contains the following arguments:
  22. ["source","java",subs="attributes,callouts,macros"]
  23. --------------------------------------------------
  24. include-tagged::{doc-tests-file}[{api}-config]
  25. --------------------------------------------------
  26. <1> The {dataframe-transform} ID
  27. <2> The source indices and query from which to gather data
  28. <3> The destination index
  29. <4> The PivotConfig
  30. <5> Optional free text description of the transform
  31. [id="{upid}-{api}-query-config"]
  32. ==== SourceConfig
  33. The indices and the query from which to collect data.
  34. If query is not set, a `match_all` query is used by default.
  35. ["source","java",subs="attributes,callouts,macros"]
  36. --------------------------------------------------
  37. include-tagged::{doc-tests-file}[{api}-source-config]
  38. --------------------------------------------------
  39. ===== QueryConfig
  40. The query with which to select data from the source.
  41. ["source","java",subs="attributes,callouts,macros"]
  42. --------------------------------------------------
  43. include-tagged::{doc-tests-file}[{api}-query-config]
  44. --------------------------------------------------
  45. ==== PivotConfig
  46. Defines the pivot function `group by` fields and the aggregation to reduce the data.
  47. ["source","java",subs="attributes,callouts,macros"]
  48. --------------------------------------------------
  49. include-tagged::{doc-tests-file}[{api}-pivot-config]
  50. --------------------------------------------------
  51. <1> The `GroupConfig` to use in the pivot
  52. <2> The aggregations to use
  53. <3> The maximum paging size for the transform when pulling data
  54. from the source. The size dynamically adjusts as the transform
  55. is running to recover from and prevent OOM issues.
  56. ===== GroupConfig
  57. The grouping terms. Defines the group by and destination fields
  58. which are produced by the pivot function. There are 3 types of
  59. groups
  60. * Terms
  61. * Histogram
  62. * Date Histogram
  63. ["source","java",subs="attributes,callouts,macros"]
  64. --------------------------------------------------
  65. include-tagged::{doc-tests-file}[{api}-group-config]
  66. --------------------------------------------------
  67. <1> The destination field
  68. <2> Group by values of the `user_id` field
  69. ===== AggregationConfig
  70. Defines the aggregations for the group fields.
  71. // TODO link to the supported aggregations
  72. ["source","java",subs="attributes,callouts,macros"]
  73. --------------------------------------------------
  74. include-tagged::{doc-tests-file}[{api}-agg-config]
  75. --------------------------------------------------
  76. <1> Aggregate the average star rating
  77. include::../execution.asciidoc[]
  78. [id="{upid}-{api}-response"]
  79. ==== Response
  80. The returned +{response}+ acknowledges the successful creation of
  81. the new {dataframe-transform} or an error if the configuration is invalid.