source-field.asciidoc 3.8 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127
  1. [[mapping-source-field]]
  2. === `_source` field
  3. The `_source` field contains the original JSON document body that was passed
  4. at index time. The `_source` field itself is not indexed (and thus is not
  5. searchable), but it is stored so that it can be returned when executing
  6. _fetch_ requests, like <<docs-get,get>> or <<search-search,search>>.
  7. If disk usage is important to you, then consider the following options:
  8. - Using <<synthetic-source,synthetic `_source`>>, which reconstructs source content at the time of retrieval instead of storing it on disk. This shrinks disk usage, at the cost of slower access to `_source` in <<docs-get,Get>> and <<search-search,Search>> queries.
  9. - <<disable-source-field,Disabling the `_source` field completely>>. This shrinks disk
  10. usage but disables features that rely on `_source`.
  11. include::synthetic-source.asciidoc[]
  12. [[disable-source-field]]
  13. ==== Disabling the `_source` field
  14. Though very handy to have around, the source field does incur storage overhead
  15. within the index. For this reason, it can be disabled as follows:
  16. [source,console]
  17. --------------------------------------------------
  18. PUT my-index-000001
  19. {
  20. "mappings": {
  21. "_source": {
  22. "enabled": false
  23. }
  24. }
  25. }
  26. --------------------------------------------------
  27. [WARNING]
  28. .Think before disabling the `_source` field
  29. ==================================================
  30. Users often disable the `_source` field without thinking about the
  31. consequences, and then live to regret it. If the `_source` field isn't
  32. available then a number of features are not supported:
  33. * The <<docs-update,`update`>>, <<docs-update-by-query,`update_by_query`>>,
  34. and <<docs-reindex,`reindex`>> APIs.
  35. * In the {kib} link:{kibana-ref}/discover.html[Discover] application, field data will not be displayed.
  36. * On the fly <<highlighting,highlighting>>.
  37. * The ability to reindex from one Elasticsearch index to another, either
  38. to change mappings or analysis, or to upgrade an index to a new major
  39. version.
  40. * The ability to debug queries or aggregations by viewing the original
  41. document used at index time.
  42. * Potentially in the future, the ability to repair index corruption
  43. automatically.
  44. ==================================================
  45. NOTE: You can't disable the `_source` field for indexes with <<index-mode-setting,`index_mode`>>
  46. set to `logsdb` or `time_series`.
  47. TIP: If disk space is a concern, rather increase the
  48. <<index-codec,compression level>> instead of disabling the `_source`.
  49. [[include-exclude]]
  50. ==== Including / Excluding fields from `_source`
  51. An expert-only feature is the ability to prune the contents of the `_source`
  52. field after the document has been indexed, but before the `_source` field is
  53. stored.
  54. WARNING: Removing fields from the `_source` has similar downsides to disabling
  55. `_source`, especially the fact that you cannot reindex documents from one
  56. Elasticsearch index to another. Consider using
  57. <<source-filtering,source filtering>> instead.
  58. The `includes`/`excludes` parameters (which also accept wildcards) can be used
  59. as follows:
  60. [source,console]
  61. --------------------------------------------------
  62. PUT logs
  63. {
  64. "mappings": {
  65. "_source": {
  66. "includes": [
  67. "*.count",
  68. "meta.*"
  69. ],
  70. "excludes": [
  71. "meta.description",
  72. "meta.other.*"
  73. ]
  74. }
  75. }
  76. }
  77. PUT logs/_doc/1
  78. {
  79. "requests": {
  80. "count": 10,
  81. "foo": "bar" <1>
  82. },
  83. "meta": {
  84. "name": "Some metric",
  85. "description": "Some metric description", <1>
  86. "other": {
  87. "foo": "one", <1>
  88. "baz": "two" <1>
  89. }
  90. }
  91. }
  92. GET logs/_search
  93. {
  94. "query": {
  95. "match": {
  96. "meta.other.foo": "one" <2>
  97. }
  98. }
  99. }
  100. --------------------------------------------------
  101. <1> These fields will be removed from the stored `_source` field.
  102. <2> We can still search on this field, even though it is not in the stored `_source`.