evaluate-data-frame.asciidoc 5.0 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118
  1. --
  2. :api: evaluate-data-frame
  3. :request: EvaluateDataFrameRequest
  4. :response: EvaluateDataFrameResponse
  5. --
  6. [role="xpack"]
  7. [id="{upid}-{api}"]
  8. === Evaluate {dfanalytics} API
  9. Evaluates the {ml} algorithm that ran on a {dataframe}.
  10. The API accepts an +{request}+ object and returns an +{response}+.
  11. [id="{upid}-{api}-request"]
  12. ==== Evaluate {dfanalytics} request
  13. ["source","java",subs="attributes,callouts,macros"]
  14. --------------------------------------------------
  15. include-tagged::{doc-tests-file}[{api}-request]
  16. --------------------------------------------------
  17. <1> Constructing a new evaluation request
  18. <2> Reference to an existing index
  19. <3> The query with which to select data from indices
  20. <4> Evaluation to be performed
  21. ==== Evaluation
  22. Evaluation to be performed.
  23. Currently, supported evaluations include: +BinarySoftClassification+, +Classification+, +Regression+.
  24. ===== Binary soft classification
  25. ["source","java",subs="attributes,callouts,macros"]
  26. --------------------------------------------------
  27. include-tagged::{doc-tests-file}[{api}-evaluation-softclassification]
  28. --------------------------------------------------
  29. <1> Constructing a new evaluation
  30. <2> Name of the field in the index. Its value denotes the actual (i.e. ground truth) label for an example. Must be either true or false.
  31. <3> Name of the field in the index. Its value denotes the probability (as per some ML algorithm) of the example being classified as positive.
  32. <4> The remaining parameters are the metrics to be calculated based on the two fields described above
  33. <5> https://en.wikipedia.org/wiki/Precision_and_recall#Precision[Precision] calculated at thresholds: 0.4, 0.5 and 0.6
  34. <6> https://en.wikipedia.org/wiki/Precision_and_recall#Recall[Recall] calculated at thresholds: 0.5 and 0.7
  35. <7> https://en.wikipedia.org/wiki/Confusion_matrix[Confusion matrix] calculated at threshold 0.5
  36. <8> https://en.wikipedia.org/wiki/Receiver_operating_characteristic#Area_under_the_curve[AuC ROC] calculated and the curve points returned
  37. ===== Classification
  38. ["source","java",subs="attributes,callouts,macros"]
  39. --------------------------------------------------
  40. include-tagged::{doc-tests-file}[{api}-evaluation-classification]
  41. --------------------------------------------------
  42. <1> Constructing a new evaluation
  43. <2> Name of the field in the index. Its value denotes the actual (i.e. ground truth) class the example belongs to.
  44. <3> Name of the field in the index. Its value denotes the predicted (as per some ML algorithm) class of the example.
  45. <4> The remaining parameters are the metrics to be calculated based on the two fields described above
  46. <5> Multiclass confusion matrix of size 3
  47. ===== Regression
  48. ["source","java",subs="attributes,callouts,macros"]
  49. --------------------------------------------------
  50. include-tagged::{doc-tests-file}[{api}-evaluation-regression]
  51. --------------------------------------------------
  52. <1> Constructing a new evaluation
  53. <2> Name of the field in the index. Its value denotes the actual (i.e. ground truth) value for an example.
  54. <3> Name of the field in the index. Its value denotes the predicted (as per some ML algorithm) value for the example.
  55. <4> The remaining parameters are the metrics to be calculated based on the two fields described above
  56. <5> https://en.wikipedia.org/wiki/Mean_squared_error[Mean squared error]
  57. <6> https://en.wikipedia.org/wiki/Coefficient_of_determination[R squared]
  58. include::../execution.asciidoc[]
  59. [id="{upid}-{api}-response"]
  60. ==== Response
  61. The returned +{response}+ contains the requested evaluation metrics.
  62. ["source","java",subs="attributes,callouts,macros"]
  63. --------------------------------------------------
  64. include-tagged::{doc-tests-file}[{api}-response]
  65. --------------------------------------------------
  66. <1> Fetching all the calculated metrics results
  67. ==== Results
  68. ===== Binary soft classification
  69. ["source","java",subs="attributes,callouts,macros"]
  70. --------------------------------------------------
  71. include-tagged::{doc-tests-file}[{api}-results-softclassification]
  72. --------------------------------------------------
  73. <1> Fetching precision metric by name
  74. <2> Fetching precision at a given (0.4) threshold
  75. <3> Fetching confusion matrix metric by name
  76. <4> Fetching confusion matrix at a given (0.5) threshold
  77. ===== Classification
  78. ["source","java",subs="attributes,callouts,macros"]
  79. --------------------------------------------------
  80. include-tagged::{doc-tests-file}[{api}-results-classification]
  81. --------------------------------------------------
  82. <1> Fetching multiclass confusion matrix metric by name
  83. <2> Fetching the contents of the confusion matrix
  84. <3> Fetching the number of classes that were not included in the matrix
  85. ===== Regression
  86. ["source","java",subs="attributes,callouts,macros"]
  87. --------------------------------------------------
  88. include-tagged::{doc-tests-file}[{api}-results-regression]
  89. --------------------------------------------------
  90. <1> Fetching mean squared error metric by name
  91. <2> Fetching the actual mean squared error value
  92. <3> Fetching R squared metric by name
  93. <4> Fetching the actual R squared value