put_data_frame.asciidoc 3.9 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119
  1. --
  2. :api: put-data-frame-transform
  3. :request: PutDataFrameTransformRequest
  4. :response: AcknowledgedResponse
  5. --
  6. [id="{upid}-{api}"]
  7. === Put Data Frame Transform API
  8. The Put Data Frame Transform API is used to create a new {dataframe-transform}.
  9. The API accepts a +{request}+ object as a request and returns a +{response}+.
  10. [id="{upid}-{api}-request"]
  11. ==== Put Data Frame Request
  12. A +{request}+ requires the following argument:
  13. ["source","java",subs="attributes,callouts,macros"]
  14. --------------------------------------------------
  15. include-tagged::{doc-tests-file}[{api}-request]
  16. --------------------------------------------------
  17. <1> The configuration of the {dataframe-job} to create
  18. [id="{upid}-{api}-config"]
  19. ==== Data Frame Transform Configuration
  20. The `DataFrameTransformConfig` object contains all the details about the {dataframe-transform}
  21. configuration and contains the following arguments:
  22. ["source","java",subs="attributes,callouts,macros"]
  23. --------------------------------------------------
  24. include-tagged::{doc-tests-file}[{api}-config]
  25. --------------------------------------------------
  26. <1> The {dataframe-transform} ID
  27. <2> The source indices and query from which to gather data
  28. <3> The destination index and optional pipeline
  29. <4> How often to check for updates to the source indices
  30. <5> The PivotConfig
  31. <6> Optional free text description of the transform
  32. [id="{upid}-{api}-query-config"]
  33. ==== SourceConfig
  34. The indices and the query from which to collect data.
  35. If query is not set, a `match_all` query is used by default.
  36. ["source","java",subs="attributes,callouts,macros"]
  37. --------------------------------------------------
  38. include-tagged::{doc-tests-file}[{api}-source-config]
  39. --------------------------------------------------
  40. ==== DestConfig
  41. The index where to write the data and the optional pipeline
  42. through which the docs should be indexed
  43. ["source","java",subs="attributes,callouts,macros"]
  44. --------------------------------------------------
  45. include-tagged::{doc-tests-file}[{api}-dest-config]
  46. --------------------------------------------------
  47. ===== QueryConfig
  48. The query with which to select data from the source.
  49. ["source","java",subs="attributes,callouts,macros"]
  50. --------------------------------------------------
  51. include-tagged::{doc-tests-file}[{api}-query-config]
  52. --------------------------------------------------
  53. ==== PivotConfig
  54. Defines the pivot function `group by` fields and the aggregation to reduce the data.
  55. ["source","java",subs="attributes,callouts,macros"]
  56. --------------------------------------------------
  57. include-tagged::{doc-tests-file}[{api}-pivot-config]
  58. --------------------------------------------------
  59. <1> The `GroupConfig` to use in the pivot
  60. <2> The aggregations to use
  61. <3> The maximum paging size for the transform when pulling data
  62. from the source. The size dynamically adjusts as the transform
  63. is running to recover from and prevent OOM issues.
  64. ===== GroupConfig
  65. The grouping terms. Defines the group by and destination fields
  66. which are produced by the pivot function. There are 3 types of
  67. groups
  68. * Terms
  69. * Histogram
  70. * Date Histogram
  71. ["source","java",subs="attributes,callouts,macros"]
  72. --------------------------------------------------
  73. include-tagged::{doc-tests-file}[{api}-group-config]
  74. --------------------------------------------------
  75. <1> The destination field
  76. <2> Group by values of the `user_id` field
  77. ===== AggregationConfig
  78. Defines the aggregations for the group fields.
  79. // TODO link to the supported aggregations
  80. ["source","java",subs="attributes,callouts,macros"]
  81. --------------------------------------------------
  82. include-tagged::{doc-tests-file}[{api}-agg-config]
  83. --------------------------------------------------
  84. <1> Aggregate the average star rating
  85. include::../execution.asciidoc[]
  86. [id="{upid}-{api}-response"]
  87. ==== Response
  88. The returned +{response}+ acknowledges the successful creation of
  89. the new {dataframe-transform} or an error if the configuration is invalid.