put_data_frame.asciidoc 4.2 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123
  1. --
  2. :api: put-data-frame-transform
  3. :request: PutDataFrameTransformRequest
  4. :response: AcknowledgedResponse
  5. --
  6. [id="{upid}-{api}"]
  7. === Put Data Frame Transform API
  8. The Put Data Frame Transform API is used to create a new {dataframe-transform}.
  9. The API accepts a +{request}+ object as a request and returns a +{response}+.
  10. [id="{upid}-{api}-request"]
  11. ==== Put Data Frame Request
  12. A +{request}+ requires the following argument:
  13. ["source","java",subs="attributes,callouts,macros"]
  14. --------------------------------------------------
  15. include-tagged::{doc-tests-file}[{api}-request]
  16. --------------------------------------------------
  17. <1> The configuration of the {dataframe-transform} to create
  18. <2> Whether or not to wait to run deferrable validations until `_start` is called.
  19. This option should be used with care as the created {dataframe-transform} will run
  20. with the privileges of the user creating it. Meaning, if they do not have privileges,
  21. such an error will not be visible until `_start` is called.
  22. [id="{upid}-{api}-config"]
  23. ==== Data Frame Transform Configuration
  24. The `DataFrameTransformConfig` object contains all the details about the {dataframe-transform}
  25. configuration and contains the following arguments:
  26. ["source","java",subs="attributes,callouts,macros"]
  27. --------------------------------------------------
  28. include-tagged::{doc-tests-file}[{api}-config]
  29. --------------------------------------------------
  30. <1> The {dataframe-transform} ID
  31. <2> The source indices and query from which to gather data
  32. <3> The destination index and optional pipeline
  33. <4> How often to check for updates to the source indices
  34. <5> The PivotConfig
  35. <6> Optional free text description of the transform
  36. [id="{upid}-{api}-query-config"]
  37. ==== SourceConfig
  38. The indices and the query from which to collect data.
  39. If query is not set, a `match_all` query is used by default.
  40. ["source","java",subs="attributes,callouts,macros"]
  41. --------------------------------------------------
  42. include-tagged::{doc-tests-file}[{api}-source-config]
  43. --------------------------------------------------
  44. ==== DestConfig
  45. The index where to write the data and the optional pipeline
  46. through which the docs should be indexed
  47. ["source","java",subs="attributes,callouts,macros"]
  48. --------------------------------------------------
  49. include-tagged::{doc-tests-file}[{api}-dest-config]
  50. --------------------------------------------------
  51. ===== QueryConfig
  52. The query with which to select data from the source.
  53. ["source","java",subs="attributes,callouts,macros"]
  54. --------------------------------------------------
  55. include-tagged::{doc-tests-file}[{api}-query-config]
  56. --------------------------------------------------
  57. ==== PivotConfig
  58. Defines the pivot function `group by` fields and the aggregation to reduce the data.
  59. ["source","java",subs="attributes,callouts,macros"]
  60. --------------------------------------------------
  61. include-tagged::{doc-tests-file}[{api}-pivot-config]
  62. --------------------------------------------------
  63. <1> The `GroupConfig` to use in the pivot
  64. <2> The aggregations to use
  65. <3> The maximum paging size for the transform when pulling data
  66. from the source. The size dynamically adjusts as the transform
  67. is running to recover from and prevent OOM issues.
  68. ===== GroupConfig
  69. The grouping terms. Defines the group by and destination fields
  70. which are produced by the pivot function. There are 3 types of
  71. groups
  72. * Terms
  73. * Histogram
  74. * Date Histogram
  75. ["source","java",subs="attributes,callouts,macros"]
  76. --------------------------------------------------
  77. include-tagged::{doc-tests-file}[{api}-group-config]
  78. --------------------------------------------------
  79. <1> The destination field
  80. <2> Group by values of the `user_id` field
  81. ===== AggregationConfig
  82. Defines the aggregations for the group fields.
  83. // TODO link to the supported aggregations
  84. ["source","java",subs="attributes,callouts,macros"]
  85. --------------------------------------------------
  86. include-tagged::{doc-tests-file}[{api}-agg-config]
  87. --------------------------------------------------
  88. <1> Aggregate the average star rating
  89. include::../execution.asciidoc[]
  90. [id="{upid}-{api}-response"]
  91. ==== Response
  92. The returned +{response}+ acknowledges the successful creation of
  93. the new {dataframe-transform} or an error if the configuration is invalid.