put-inference.asciidoc 15 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622
  1. [role="xpack"]
  2. [testenv="basic"]
  3. [[put-inference]]
  4. === Create {infer} trained model API
  5. [subs="attributes"]
  6. ++++
  7. <titleabbrev>Create {infer} trained model</titleabbrev>
  8. ++++
  9. Creates an {infer} trained model.
  10. +
  11. --
  12. WARNING: Models created in version 7.8.0 are not backwards compatible
  13. with older node versions. If in a mixed cluster environment,
  14. all nodes must be at least 7.8.0 to use a model stored by
  15. a 7.8.0 node.
  16. --
  17. experimental[]
  18. [[ml-put-inference-request]]
  19. ==== {api-request-title}
  20. `PUT _ml/inference/<model_id>`
  21. [[ml-put-inference-prereq]]
  22. ==== {api-prereq-title}
  23. If the {es} {security-features} are enabled, you must have the following
  24. built-in roles and privileges:
  25. * `machine_learning_admin`
  26. For more information, see <<security-privileges>> and <<built-in-roles>>.
  27. [[ml-put-inference-desc]]
  28. ==== {api-description-title}
  29. The create {infer} trained model API enables you to supply a trained model that
  30. is not created by {dfanalytics}.
  31. [[ml-put-inference-path-params]]
  32. ==== {api-path-parms-title}
  33. `<model_id>`::
  34. (Required, string)
  35. include::{docdir}/ml/ml-shared.asciidoc[tag=model-id]
  36. [role="child_attributes"]
  37. [[ml-put-inference-request-body]]
  38. ==== {api-request-body-title}
  39. `compressed_definition`::
  40. (Required, string)
  41. The compressed (GZipped and Base64 encoded) {infer} definition of the model.
  42. If `compressed_definition` is specified, then `definition` cannot be specified.
  43. //Begin definition
  44. `definition`::
  45. (Required, object)
  46. The {infer} definition for the model. If `definition` is specified, then
  47. `compressed_definition` cannot be specified.
  48. +
  49. .Properties of `definition`
  50. [%collapsible%open]
  51. ====
  52. //Begin preprocessors
  53. `preprocessors`::
  54. (Optional, object)
  55. Collection of preprocessors. See <<ml-put-inference-preprocessor-example>>.
  56. +
  57. .Properties of `preprocessors`
  58. [%collapsible%open]
  59. =====
  60. //Begin frequency encoding
  61. `frequency_encoding`::
  62. (Required, object)
  63. Defines a frequency encoding for a field.
  64. +
  65. .Properties of `frequency_encoding`
  66. [%collapsible%open]
  67. ======
  68. `feature_name`::
  69. (Required, string)
  70. The name of the resulting feature.
  71. `field`::
  72. (Required, string)
  73. The field name to encode.
  74. `frequency_map`::
  75. (Required, object map of string:double)
  76. Object that maps the field value to the frequency encoded value.
  77. ======
  78. //End frequency encoding
  79. //Begin one hot encoding
  80. `one_hot_encoding`::
  81. (Required, object)
  82. Defines a one hot encoding map for a field.
  83. +
  84. .Properties of `one_hot_encoding`
  85. [%collapsible%open]
  86. ======
  87. `field`::
  88. (Required, string)
  89. The field name to encode.
  90. `hot_map`::
  91. (Required, object map of strings)
  92. String map of "field_value: one_hot_column_name".
  93. ======
  94. //End one hot encoding
  95. //Begin target mean encoding
  96. `target_mean_encoding`::
  97. (Required, object)
  98. Defines a target mean encoding for a field.
  99. +
  100. .Properties of `target_mean_encoding`
  101. [%collapsible%open]
  102. ======
  103. `default_value`:::
  104. (Required, double)
  105. The feature value if the field value is not in the `target_map`.
  106. `feature_name`:::
  107. (Required, string)
  108. The name of the resulting feature.
  109. `field`:::
  110. (Required, string)
  111. The field name to encode.
  112. `target_map`:::
  113. (Required, object map of string:double)
  114. Object that maps the field value to the target mean value.
  115. ======
  116. //End target mean encoding
  117. =====
  118. //End preprocessors
  119. //Begin trained model
  120. `trained_model`::
  121. (Required, object)
  122. The definition of the trained model.
  123. +
  124. .Properties of `trained_model`
  125. [%collapsible%open]
  126. =====
  127. //Begin tree
  128. `tree`::
  129. (Required, object)
  130. The definition for a binary decision tree.
  131. +
  132. .Properties of `tree`
  133. [%collapsible%open]
  134. ======
  135. `classification_labels`:::
  136. (Optional, string) An array of classification labels (used for
  137. `classification`).
  138. `feature_names`:::
  139. (Required, string)
  140. Features expected by the tree, in their expected order.
  141. `target_type`:::
  142. (Required, string)
  143. String indicating the model target type; `regression` or `classification`.
  144. `tree_structure`:::
  145. (Required, object)
  146. An array of `tree_node` objects. The nodes must be in ordinal order by their
  147. `tree_node.node_index` value.
  148. ======
  149. //End tree
  150. //Begin tree node
  151. `tree_node`::
  152. (Required, object)
  153. The definition of a node in a tree.
  154. +
  155. --
  156. There are two major types of nodes: leaf nodes and not-leaf nodes.
  157. * Leaf nodes only need `node_index` and `leaf_value` defined.
  158. * All other nodes need `split_feature`, `left_child`, `right_child`,
  159. `threshold`, `decision_type`, and `default_left` defined.
  160. --
  161. +
  162. .Properties of `tree_node`
  163. [%collapsible%open]
  164. ======
  165. `decision_type`::
  166. (Optional, string)
  167. Indicates the positive value (in other words, when to choose the left node)
  168. decision type. Supported `lt`, `lte`, `gt`, `gte`. Defaults to `lte`.
  169. `default_left`::
  170. (Optional, boolean)
  171. Indicates whether to default to the left when the feature is missing. Defaults
  172. to `true`.
  173. `leaf_value`::
  174. (Optional, double)
  175. The leaf value of the of the node, if the value is a leaf (in other words, no
  176. children).
  177. `left_child`::
  178. (Optional, integer)
  179. The index of the left child.
  180. `node_index`::
  181. (Integer)
  182. The index of the current node.
  183. `right_child`::
  184. (Optional, integer)
  185. The index of the right child.
  186. `split_feature`::
  187. (Optional, integer)
  188. The index of the feature value in the feature array.
  189. `split_gain`::
  190. (Optional, double) The information gain from the split.
  191. `threshold`::
  192. (Optional, double)
  193. The decision threshold with which to compare the feature value.
  194. ======
  195. //End tree node
  196. //Begin ensemble
  197. `ensemble`::
  198. (Optional, object)
  199. The definition for an ensemble model. See <<ml-put-inference-model-example>>.
  200. +
  201. .Properties of `ensemble`
  202. [%collapsible%open]
  203. ======
  204. //Begin aggregate output
  205. `aggregate_output`::
  206. (Required, object)
  207. An aggregated output object that defines how to aggregate the outputs of the
  208. `trained_models`. Supported objects are `weighted_mode`, `weighted_sum`, and
  209. `logistic_regression`. See <<ml-put-inference-aggregated-output-example>>.
  210. +
  211. .Properties of `aggregate_output`
  212. [%collapsible%open]
  213. =======
  214. //Begin logistic regression
  215. `logistic_regression`::
  216. (Optional, object)
  217. This `aggregated_output` type works with binary classification (classification
  218. for values [0, 1]). It multiplies the outputs (in the case of the `ensemble`
  219. model, the inference model values) by the supplied `weights`. The resulting
  220. vector is summed and passed to a
  221. https://en.wikipedia.org/wiki/Sigmoid_function[`sigmoid` function]. The result
  222. of the `sigmoid` function is considered the probability of class 1 (`P_1`),
  223. consequently, the probability of class 0 is `1 - P_1`. The class with the
  224. highest probability (either 0 or 1) is then returned. For more information about
  225. logistic regression, see
  226. https://en.wikipedia.org/wiki/Logistic_regression[this wiki article].
  227. +
  228. .Properties of `logistic_regression`
  229. [%collapsible%open]
  230. ========
  231. `weights`:::
  232. (Required, double)
  233. The weights to multiply by the input values (the inference values of the trained
  234. models).
  235. ========
  236. //End logistic regression
  237. //Begin weighted sum
  238. `weighted_sum`::
  239. (Optional, object)
  240. This `aggregated_output` type works with regression. The weighted sum of the
  241. input values.
  242. +
  243. .Properties of `weighted_sum`
  244. [%collapsible%open]
  245. ========
  246. `weights`:::
  247. (Required, double)
  248. The weights to multiply by the input values (the inference values of the trained
  249. models).
  250. ========
  251. //End weighted sum
  252. //Begin weighted mode
  253. `weighted_mode`::
  254. (Optional, object)
  255. This `aggregated_output` type works with regression or classification. It takes
  256. a weighted vote of the input values. The most common input value (taking the
  257. weights into account) is returned.
  258. +
  259. .Properties of `weighted_mode`
  260. [%collapsible%open]
  261. ========
  262. `weights`:::
  263. (Required, double)
  264. The weights to multiply by the input values (the inference values of the trained
  265. models).
  266. ========
  267. //End weighted mode
  268. =======
  269. //End aggregate output
  270. `classification_labels`::
  271. (Optional, string)
  272. An array of classification labels.
  273. `feature_names`::
  274. (Optional, string)
  275. Features expected by the ensemble, in their expected order.
  276. `target_type`::
  277. (Required, string)
  278. String indicating the model target type; `regression` or `classification.`
  279. `trained_models`::
  280. (Required, object)
  281. An array of `trained_model` objects. Supported trained models are `tree` and
  282. `ensemble`.
  283. ======
  284. //End ensemble
  285. =====
  286. //End trained model
  287. ====
  288. //End definition
  289. `description`::
  290. (Optional, string)
  291. A human-readable description of the {infer} trained model.
  292. //Begin inference_config
  293. `inference_config`::
  294. (Required, object)
  295. The default configuration for inference. This can be either a `regression`
  296. or `classification` configuration. It must match the underlying
  297. `definition.trained_model`'s `target_type`.
  298. +
  299. .Properties of `inference_config`
  300. [%collapsible%open]
  301. ====
  302. `regression`:::
  303. (Optional, object)
  304. Regression configuration for inference.
  305. +
  306. .Properties of regression inference
  307. [%collapsible%open]
  308. =====
  309. `num_top_feature_importance_values`::::
  310. (Optional, integer)
  311. include::{docdir}/ml/ml-shared.asciidoc[tag=inference-config-regression-num-top-feature-importance-values]
  312. `results_field`::::
  313. (Optional, string)
  314. include::{docdir}/ml/ml-shared.asciidoc[tag=inference-config-results-field]
  315. =====
  316. `classification`:::
  317. (Optional, object)
  318. Classification configuration for inference.
  319. +
  320. .Properties of classification inference
  321. [%collapsible%open]
  322. =====
  323. `num_top_classes`::::
  324. (Optional, integer)
  325. include::{docdir}/ml/ml-shared.asciidoc[tag=inference-config-classification-num-top-classes]
  326. `num_top_feature_importance_values`::::
  327. (Optional, integer)
  328. include::{docdir}/ml/ml-shared.asciidoc[tag=inference-config-classification-num-top-feature-importance-values]
  329. `prediction_field_type`::::
  330. (Optional, string)
  331. include::{docdir}/ml/ml-shared.asciidoc[tag=inference-config-classification-prediction-field-type]
  332. `results_field`::::
  333. (Optional, string)
  334. include::{docdir}/ml/ml-shared.asciidoc[tag=inference-config-results-field]
  335. `top_classes_results_field`::::
  336. (Optional, string)
  337. include::{docdir}/ml/ml-shared.asciidoc[tag=inference-config-classification-top-classes-results-field]
  338. =====
  339. ====
  340. //End of inference_config
  341. //Begin input
  342. `input`::
  343. (Required, object)
  344. The input field names for the model definition.
  345. +
  346. .Properties of `input`
  347. [%collapsible%open]
  348. ====
  349. `field_names`:::
  350. (Required, string)
  351. An array of input field names for the model.
  352. ====
  353. //End input
  354. `metadata`::
  355. (Optional, object)
  356. An object map that contains metadata about the model.
  357. `tags`::
  358. (Optional, string)
  359. An array of tags to organize the model.
  360. [[ml-put-inference-example]]
  361. ==== {api-examples-title}
  362. [[ml-put-inference-preprocessor-example]]
  363. ===== Preprocessor examples
  364. The example below shows a `frequency_encoding` preprocessor object:
  365. [source,js]
  366. ----------------------------------
  367. {
  368. "frequency_encoding":{
  369. "field":"FlightDelayType",
  370. "feature_name":"FlightDelayType_frequency",
  371. "frequency_map":{
  372. "Carrier Delay":0.6007414737092798,
  373. "NAS Delay":0.6007414737092798,
  374. "Weather Delay":0.024573576178086153,
  375. "Security Delay":0.02476631010889467,
  376. "No Delay":0.6007414737092798,
  377. "Late Aircraft Delay":0.6007414737092798
  378. }
  379. }
  380. }
  381. ----------------------------------
  382. //NOTCONSOLE
  383. The next example shows a `one_hot_encoding` preprocessor object:
  384. [source,js]
  385. ----------------------------------
  386. {
  387. "one_hot_encoding":{
  388. "field":"FlightDelayType",
  389. "hot_map":{
  390. "Carrier Delay":"FlightDelayType_Carrier Delay",
  391. "NAS Delay":"FlightDelayType_NAS Delay",
  392. "No Delay":"FlightDelayType_No Delay",
  393. "Late Aircraft Delay":"FlightDelayType_Late Aircraft Delay"
  394. }
  395. }
  396. }
  397. ----------------------------------
  398. //NOTCONSOLE
  399. This example shows a `target_mean_encoding` preprocessor object:
  400. [source,js]
  401. ----------------------------------
  402. {
  403. "target_mean_encoding":{
  404. "field":"FlightDelayType",
  405. "feature_name":"FlightDelayType_targetmean",
  406. "target_map":{
  407. "Carrier Delay":39.97465788139886,
  408. "NAS Delay":39.97465788139886,
  409. "Security Delay":203.171206225681,
  410. "Weather Delay":187.64705882352948,
  411. "No Delay":39.97465788139886,
  412. "Late Aircraft Delay":39.97465788139886
  413. },
  414. "default_value":158.17995752420433
  415. }
  416. }
  417. ----------------------------------
  418. //NOTCONSOLE
  419. [[ml-put-inference-model-example]]
  420. ===== Model examples
  421. The first example shows a `trained_model` object:
  422. [source,js]
  423. ----------------------------------
  424. {
  425. "tree":{
  426. "feature_names":[
  427. "DistanceKilometers",
  428. "FlightTimeMin",
  429. "FlightDelayType_NAS Delay",
  430. "Origin_targetmean",
  431. "DestRegion_targetmean",
  432. "DestCityName_targetmean",
  433. "OriginAirportID_targetmean",
  434. "OriginCityName_frequency",
  435. "DistanceMiles",
  436. "FlightDelayType_Late Aircraft Delay"
  437. ],
  438. "tree_structure":[
  439. {
  440. "decision_type":"lt",
  441. "threshold":9069.33437193022,
  442. "split_feature":0,
  443. "split_gain":4112.094574306927,
  444. "node_index":0,
  445. "default_left":true,
  446. "left_child":1,
  447. "right_child":2
  448. },
  449. ...
  450. {
  451. "node_index":9,
  452. "leaf_value":-27.68987349695448
  453. },
  454. ...
  455. ],
  456. "target_type":"regression"
  457. }
  458. }
  459. ----------------------------------
  460. //NOTCONSOLE
  461. The following example shows an `ensemble` model object:
  462. [source,js]
  463. ----------------------------------
  464. "ensemble":{
  465. "feature_names":[
  466. ...
  467. ],
  468. "trained_models":[
  469. {
  470. "tree":{
  471. "feature_names":[],
  472. "tree_structure":[
  473. {
  474. "decision_type":"lte",
  475. "node_index":0,
  476. "leaf_value":47.64069875778043,
  477. "default_left":false
  478. }
  479. ],
  480. "target_type":"regression"
  481. }
  482. },
  483. ...
  484. ],
  485. "aggregate_output":{
  486. "weighted_sum":{
  487. "weights":[
  488. ...
  489. ]
  490. }
  491. },
  492. "target_type":"regression"
  493. }
  494. ----------------------------------
  495. //NOTCONSOLE
  496. [[ml-put-inference-aggregated-output-example]]
  497. ===== Aggregated output example
  498. Example of a `logistic_regression` object:
  499. [source,js]
  500. ----------------------------------
  501. "aggregate_output" : {
  502. "logistic_regression" : {
  503. "weights" : [2.0, 1.0, .5, -1.0, 5.0, 1.0, 1.0]
  504. }
  505. }
  506. ----------------------------------
  507. //NOTCONSOLE
  508. Example of a `weighted_sum` object:
  509. [source,js]
  510. ----------------------------------
  511. "aggregate_output" : {
  512. "weighted_sum" : {
  513. "weights" : [1.0, -1.0, .5, 1.0, 5.0]
  514. }
  515. }
  516. ----------------------------------
  517. //NOTCONSOLE
  518. Example of a `weighted_mode` object:
  519. [source,js]
  520. ----------------------------------
  521. "aggregate_output" : {
  522. "weighted_mode" : {
  523. "weights" : [1.0, 1.0, 1.0, 1.0, 1.0]
  524. }
  525. }
  526. ----------------------------------
  527. //NOTCONSOLE
  528. [[ml-put-inference-json-schema]]
  529. ===== {infer-cap} JSON schema
  530. For the full JSON schema of model {infer},
  531. https://github.com/elastic/ml-json-schemas[click here].