Browse Source

[DOCS] Add release notes for 8.5.0 (#90394) (#90485)

Forward port https://github.com/elastic/elasticsearch/pull/90394 to main
branch

> Added release notes and highlights for release 8.5.0 >  > Relates to
#90202 and #90201 >
Christos Soulios 3 years ago
parent
commit
1c0f064599

+ 274 - 1
docs/reference/release-notes/8.5.0.asciidoc

@@ -3,4 +3,277 @@
 
 coming[8.5.0]
 
-Also see <<breaking-changes-8.5,Breaking changes in 8.5>>.
+Also see <<breaking-changes-8.5,Breaking changes in 8.5>>.
+
+[[breaking-8.5.0]]
+[float]
+=== Breaking changes
+
+CRUD::
+* Reject unknown bulk actions {es-pull}89450[#89450]
+
+[[bug-8.5.0]]
+[float]
+=== Bug fixes
+
+Aggregations::
+* Fix `auto_date_histogram` > `ip_range` {es-pull}90317[#90317] (issue: {es-issue}90121[#90121])
+* Fail when rebuilding scorer in `breadth_first` mode and query context has changed {es-pull}89993[#89993] (issue: {es-issue}37650[#37650])
+* Fix merging with empty results {es-pull}86939[#86939] (issue: {es-issue}84622[#84622])
+* Fix partial reduce bug in `ip_prefix` {es-pull}89734[#89734] (issue: {es-issue}89686[#89686])
+* Remove unexpected meta param in agg's response {es-pull}89467[#89467] (issue: {es-issue}89455[#89455])
+* Top_metrics: consistently allow sorting by size>1 {es-pull}89974[#89974] (issue: {es-issue}86663[#86663])
+* check parent circuit breaker when allocating empty bucket {es-pull}89568[#89568] (issue: {es-issue}80789[#80789])
+
+Authorization::
+* Remove magic string for the empty Role {es-pull}89766[#89766]
+
+Cluster Coordination::
+* Capture deprecation warnings in batched master tasks {es-pull}85525[#85525] (issue: {es-issue}85506[#85506])
+* Check circuit breaker before sending join request {es-pull}89318[#89318] (issue: {es-issue}85003[#85003])
+
+Engine::
+* fix "path.conf'" typo in Security.java {es-pull}89248[#89248] (issue: {es-issue}89327[#89327])
+
+Geo::
+* Buffer H3Polygon2D bounding boxes to avoid edge precision issues {es-pull}89196[#89196] (issues: {es-issue}89868[#89868], {es-issue}87391[#87391])
+* Date histogram range edge case fix {es-pull}88957[#88957]
+* Format runtime `geo_points` {es-pull}85449[#85449] (issue: {es-issue}85245[#85245])
+* Synthetic _source error on script loads {es-pull}88334[#88334]
+
+Graph::
+* Graph: fix race condition in timeout {es-pull}88946[#88946] (issue: {es-issue}55396[#55396])
+
+Health::
+* Disk indicator change details {es-pull}90189[#90189]
+* Fix disk indicator impacts and diagnosis {es-pull}90262[#90262]
+* Update min version for health node reporting to 8.5 {es-pull}90365[#90365] (issue: {es-issue}90359[#90359])
+
+Indices APIs::
+* Fix shard splitting for `nested` {es-pull}89351[#89351] (issue: {es-issue}88109[#88109])
+
+Infra/Core::
+* Fix `allow_no_indices` request option in special cases {es-pull}89622[#89622]
+* Fix repeated error save loops in File Settings Service {es-pull}90271[#90271] (issue: {es-issue}90222[#90222])
+* Support camel case dates on 7.x indices {es-pull}88914[#88914] (issue: {es-issue}84199[#84199])
+
+Machine Learning::
+* Return 408 instead of 500 when open/start APIs time out {es-pull}89775[#89775] (issue: {es-issue}89585[#89585])
+
+Mapping::
+* Fix duplication bug for source fallback in numeric types {es-pull}89352[#89352]
+* Include runtime fields in total fields count {es-pull}89251[#89251] (issue: {es-issue}88265[#88265])
+* Fix `aggregate_metric_double` multi-values exception {es-pull}90290[#90290]
+
+Search::
+* Add support for predefined char class regexp on wildcard fields {es-pull}90064[#90064]
+* Deduplicate fetching doc-values fields {es-pull}89094[#89094]
+* Safeguard `RegExp` use against `StackOverflowError` {es-pull}84624[#84624] (issue: {es-issue}82923[#82923])
+
+Snapshot/Restore::
+* Fix incorrect failed shards count in APIs for current snapshots {es-pull}89534[#89534]
+* Fix over-allocation of mounted indices on a cold/frozen node {es-pull}86331[#86331]
+* Make sure listener is resolved when file queue is cleared {es-pull}89929[#89929]
+* Re-registering corrupt repository unblocks it {es-pull}89719[#89719] (issue: {es-issue}89130[#89130])
+* Reject unknown request body fields in Mount API {es-pull}88987[#88987] (issue: {es-issue}75982[#75982])
+
+TSDS::
+* Fix segment stats in tsdb {es-pull}89754[#89754] (issue: {es-issue}89609[#89609])
+* Synthetic _source: fix extra fields in GET {es-pull}89778[#89778]
+* Synthetic _source: fix `scaled_float` rounding {es-pull}88916[#88916] (issue: {es-issue}88854[#88854])
+
+Transform::
+* Fix NPE in transform scheduling {es-pull}90347[#90347] (issues: {es-issue}90356[#90356], {es-issue}88203[#88203], {es-issue}90301[#90301], {es-issue}90255[#90255])
+* Improve error handling in state persistence {es-pull}88910[#88910] (issue: {es-issue}88905[#88905])
+* Return 408 instead of 500 when start api times out {es-pull}89774[#89774]
+
+Vector Search::
+* Fix bug for kNN with filtered aliases {es-pull}89621[#89621]
+
+[[deprecation-8.5.0]]
+[float]
+=== Deprecations
+
+Infra/Plugins::
+* Deprecate network plugins {es-pull}88924[#88924]
+* Deprecate overriding `DiscoveryPlugin` internals {es-pull}88925[#88925]
+
+[[enhancement-8.5.0]]
+[float]
+=== Enhancements
+
+Authentication::
+* More accurate error message for LDAP user modes {es-pull}89492[#89492]
+
+Authorization::
+* Add indices permissions to Enterprise Search service account {es-pull}89869[#89869]
+* Add info of resolved roles in denial messages {es-pull}89680[#89680]
+
+Autoscaling::
+* Centralize the concept of processors configuration {es-pull}89662[#89662]
+
+Cluster Coordination::
+* Preemptively compute `RoutingNodes` and the indices lookup during publication {es-pull}89005[#89005]
+* Preemptively initialize routing nodes and indices lookup on all node types {es-pull}89032[#89032]
+
+Distributed::
+* Batch index delete cluster state updates {es-pull}90033[#90033] (issue: {es-issue}90022[#90022])
+* Increase the minimum size of the management pool to 2 {es-pull}90193[#90193]
+
+Health::
+* Add IDs to health API diagnoses and impacts {es-pull}90072[#90072]
+* Adding a check to the master stability health API when there is no master and the current node is not master eligible {es-pull}89219[#89219]
+* Adding logic to `master_is_stable` indicator to check for discovery problems {es-pull}88020[#88020]
+* Polling for cluster diagnostics information {es-pull}89014[#89014]
+* Update SLM health diagnosis message to include unhealthy policy details {es-pull}89138[#89138]
+
+Highlighting::
+* Improve efficiency of `BoundedBreakIteratorScanner` fragmentation algorithm {es-pull}89041[#89041] (issues: {es-issue}73569[#73569], {es-issue}73785[#73785])
+
+ILM+SLM::
+* Add validations for the downsampling ILM action {es-pull}90295[#90295]
+* ILM must not rollover empty indices {es-pull}89557[#89557] (issue: {es-issue}86203[#86203])
+* Reuse Info in lifecycle step {es-pull}89419[#89419]
+* move log-related logic into log block in IndexLifecycleRunner {es-pull}89292[#89292]
+
+Infra/Core::
+* Add reserved snapshot/repo action {es-pull}89601[#89601]
+* Fleet: Add `upgrade_status` attributes to agents {es-pull}89845[#89845]
+* Operator/autoscaling {es-pull}89708[#89708]
+* Operator/role mapping {es-pull}89667[#89667]
+* Operator/slm policy {es-pull}89567[#89567]
+
+Infra/Node Lifecycle::
+* Distinguish no shutdowns case in `NodeShutdownAllocationDecider` {es-pull}89851[#89851] (issue: {es-issue}89823[#89823])
+
+Infra/Plugins::
+* Add deprecation message for deprecated plugin APIs {es-pull}88961[#88961]
+* Register stable plugins in `ActionModule` {es-pull}90067[#90067]
+* [Stable plugin API] Load plugin named components {es-pull}89969[#89969]
+
+Infra/Scripting::
+* Initial code to support binary expression scripts {es-pull}89895[#89895]
+* Script: Protected `_source` inside update scripts {es-pull}88733[#88733]
+* Script: Reindex & `UpdateByQuery` Metadata {es-pull}88665[#88665]
+* Script: Write Field API `NestedDocument` support {es-pull}90021[#90021]
+* Script: Write Field API path manipulation {es-pull}89889[#89889]
+* Script: Write Field API with basic path resolution {es-pull}89738[#89738]
+* Script: Write Fields API for reindex, update, update by query {es-pull}90145[#90145]
+
+Infra/Settings::
+* Introduce max headroom for disk watermark stages {es-pull}88639[#88639] (issue: {es-issue}81406[#81406])
+
+License::
+* License check for user profile collaboration feature {es-pull}89990[#89990]
+
+Machine Learning::
+* Add new `text_similarity` nlp task {es-pull}88439[#88439]
+* Add new trained model deployment cache clear API {es-pull}89074[#89074]
+* Add processor autoscaling decider {es-pull}89645[#89645]
+* Distribute trained model allocations across availability zones {es-pull}89822[#89822]
+* Frequent Items: use a bitset for deduplication {es-pull}88943[#88943]
+* Optimize frequent items transaction lookup {es-pull}89062[#89062]
+* Return 408 when start deployment api times out {es-pull}89612[#89612]
+* Skip renormalization after node shutdown API called {es-pull}89347[#89347]
+
+Mapping::
+* Add `synthetic_source` support to `aggregate_metric_double` fields {es-pull}88909[#88909]
+* Add source fallback for keyword fields using operation {es-pull}88735[#88735]
+* Add source fallback support for `match_only_text` mapped type {es-pull}89473[#89473]
+* Add source fallback support for date and `date_nanos` mapped types {es-pull}89440[#89440]
+* Add source fallback support for unsigned long mapped type {es-pull}89349[#89349]
+* Add support for source fallback with scaled float field type {es-pull}89053[#89053]
+* Add support for source fallback with the boolean field type {es-pull}89052[#89052]
+* Add text field support in the Painless scripting fields API {es-pull}89396[#89396]
+* Clarified fielddata not supported for text fields error message {es-pull}89770[#89770] (issue: {es-issue}89485[#89485])
+* Fleet: Add new mappings for .fleet-agents `last_checkin_message` and components fields {es-pull}89599[#89599]
+* Support source fallback for byte, short, and long fields {es-pull}88954[#88954]
+* Support source fallback for double, float, and `half_float` field types {es-pull}89010[#89010]
+
+Network::
+* Use chunked REST serialization for large REST responses {es-pull}88311[#88311]
+
+Recovery::
+* Do not use recovery from snapshots in searchable snapshots {es-pull}86388[#86388]
+
+SQL::
+* Implement DATE_FORMAT function {es-pull}88388[#88388] (issue: {es-issue}55065[#55065])
+* Set `track_total_hits` to false when not needed {es-pull}89106[#89106] (issue: {es-issue}88764[#88764])
+
+Search::
+* Enable `BloomFilter` for `_id` of non-datastream indices {es-pull}88409[#88409]
+* In the field capabilities API, re-add support for fields in the request body {es-pull}88972[#88972] (issue: {es-issue}86875[#86875])
+
+Security::
+* Add usage stats report for user profiles {es-pull}90123[#90123]
+* Grace period for user profile activation {es-pull}89566[#89566]
+* Limited-by role descriptors in Get/QueryApiKey response {es-pull}89273[#89273]
+* Option to return profile uid in `GetUser` response {es-pull}89570[#89570]
+* Return 400 error for `GetUserPrivileges` call with API keys {es-pull}89333[#89333]
+* Show assigned role descriptors in Get/QueryApiKey response {es-pull}89166[#89166]
+* User Profile - Detailed errors in `hasPrivileges` response {es-pull}89224[#89224]
+* User Profile - `GetProfile` API nows supports multiple UIDs {es-pull}89023[#89023]
+
+Snapshot/Restore::
+* Make `SnapshotsInProgress` Diffable {es-pull}89619[#89619] (issue: {es-issue}88732[#88732])
+* Prioritize shard snapshot tasks over file snapshot tasks and limit the number of the concurrently running snapshot tasks {es-pull}88209[#88209] (issue: {es-issue}83408[#83408])
+
+Stats::
+* Introduce node mappings stats {es-pull}89807[#89807]
+
+TSDS::
+* Synthetic _source: support `match_only_text` {es-pull}89516[#89516]
+* Synthetic _source: support histogram field {es-pull}89833[#89833]
+* Synthetic _source: support version field type {es-pull}89706[#89706]
+* TSDB: Build `_id` without reparsing {es-pull}88789[#88789]
+* [TSDB] Metric fields in the field caps API {es-pull}88695[#88695]
+
+Transform::
+* Add an unattended mode setting to transform {es-pull}89212[#89212]
+
+[[feature-8.5.0]]
+[float]
+=== New features
+
+Authorization::
+* Introduce the new `read_security` cluster privilege {es-pull}89790[#89790] (issue: {es-issue}89245[#89245])
+
+Health::
+* Enable the health node and the disk health indicator #84811 {es-pull}90085[#90085]
+
+Infra/Core::
+* Provide tracing implementation using OpenTelemetry and APM Java agent {es-pull}88443[#88443] (issue: {es-issue}84369[#84369])
+
+Infra/Plugins::
+* Stable Plugin API module and analysis interfaces {es-pull}88775[#88775]
+
+Machine Learning::
+* Make `bucket_correlation` aggregation generally available {es-pull}88655[#88655]
+* Make `bucket_count_ks_test` aggregation generally available {es-pull}88657[#88657]
+
+Security::
+* Support bulk updates of API keys {es-pull}88856[#88856]
+
+TSDB::
+* Add a TSID global ordinal to `TimeSeriesIndexSearcher` {es-pull}90035[#90035]
+* Release time-series (TSDB) functionality {es-pull}90116[#90116] (issue: {es-issue}74660[#74660])
+* Synthetic _source: support `ignore_above` {es-pull}89466[#89466]
+
+Vector Search::
+* Synthetic _source: support `dense_vector` {es-pull}89840[#89840]
+
+[[regression-8.5.0]]
+[float]
+=== Regressions
+
+Infra/Scripting::
+* Fix Fields API Caching Regression {es-pull}90017[#90017]
+
+[[upgrade-8.5.0]]
+[float]
+=== Upgrades
+
+Client::
+* Upgrade Apache Commons Logging to 1.2 {es-pull}85745[#85745] (issue: {es-issue}40305[#40305])
+
+

+ 83 - 6
docs/reference/release-notes/highlights.asciidoc

@@ -18,13 +18,90 @@ Other versions:
 | {ref-bare}/8.1/release-highlights.html[8.1]
 | {ref-bare}/8.0/release-highlights.html[8.0]
 
-// The notable-highlights tag marks entries that
-// should be featured in the Stack Installation and Upgrade Guide:
 // tag::notable-highlights[]
-// [discrete]
-// === Heading
-//
-// Description.
+
+[discrete]
+[[speed_up_sql_queries_by_not_tracking_total_hits_by_default]]
+=== Speed up SQL queries by not tracking total hits by default
+SQL query translator now explicitly sets track_total_hits to false when
+not needed.
+This has a significant impact on SQL query performance in cases where total hits
+are not needed to calculate the final result, in particular when the cost of evaluation
+of a single document is particularly high (eg. queries that involve script evaluation)
+and in queries with a small LIMIT value.
+In our tests, on some specific queries, we see a speed-up of more than 50%,
+with peaks of ~95% (from 600ms to 20ms).
+
+{es-pull}89106[#89106]
+
+[discrete]
+[[ilm_no_longer_rolls_over_empty_indices]]
+=== ILM no longer rolls over empty indices
+For both new and existing Index Lifecycle Management (ILM) policies,
+the rollover action will only execute if an index has at least one document.
+
+For indices with a `max_age` condition that are no longer being written
+to, this will mean that they will no longer roll over every time their
+`max_age` is reached.
+
+A policy can override this behavior, and explicitly opt in to rolling over
+empty indices, by adding a `"min_docs": 0` condition:
+
+[source,console]
+----
+PUT _ilm/policy/allow_empty_rollover_policy
+{
+  "policy": {
+    "phases": {
+      "hot": {
+        "actions": {
+          "rollover" : {
+            "max_age": "7d",
+            "max_size": "100gb",
+            "min_docs": 0
+          }
+        }
+      }
+    }
+  }
+}
+----
+
+This can also be disabled on a cluster-wide basis by setting
+`indices.lifecycle.rollover.only_if_has_documents` to `false`.
+
+{es-pull}89557[#89557]
+
+[discrete]
+[[release_time_series_data_stream_tsds_functionality]]
+=== Release time series data stream (TSDS) functionality
+Elasticsearch offers support for time series data stream (TSDS) indices.
+A TSDS index is an index that contains time series metrics data as part
+of a data stream. Elasticsearch routes the incoming documents into a TSDS
+index so that all the documents for a particular time series are on the
+same shard, and then sorts the shard by time series and timestamp. This
+structure has a few advantages:
+
+1. Documents from the same time series are next to each other on the shard, and
+hence stored next to each other on the disk, so the operating system pages are
+much more homogeneous and compress better, yielding massive reduction in TCO.
+
+2. The analysis of a time series typically involves comparing each two consecutive
+docs (samples), examining the last doc in a given time window, etc., which is quite
+complex when the next doc could be on any shard, and in fact on any index. Sorting
+by time series and timestamp allows improved analysis, both in terms of performance
+and in terms of our ability to add new aggregations.
+
+Finally,  as part of the Index Lifecycle Management of metrics data time series,
+Elasticsearch enables a Downsampling action. When an index is downsampled,
+Elasticsearch keeps a single document with statistical summaries per each bucket
+of time in the time series. Supported aggregations can then be run on the data
+stream and include both downsampled indices and raw data indices, without the
+user needing to be aware of that. Downsampling of downsampled indices, to more
+coarse time resolution, is also supported.
+
+{es-pull}90116[#90116]
+
 // end::notable-highlights[]