8.4 release notes ported from 8.3.0 release (#88136)

* Generate release notes for v8.3.0 (#87294)

* Generate release notes for v8.3.0

* [DOCS] Add 8.3 migration file to index

* Fixed version number

* Fix formatting of deprecation in 85326

* Use asciidoc format for deprecations

* Extract static content from migration/index

* This was just an enhancement

* Nope, it was an upgrade

* Added migration/index.asciidoc generation support (#87318)

Including extracting static content from migration/index, so the template would be as light as possible.

The reason for this work is because the gradle task `generateReleaseNotes` was not correctly adding new links and imports to the migrations/index and that caused documentation to fail building for 8.3.0.

* [DOCS] Add ml-cpp PRs to release notes

* Added back incorrectly deleted changlog

* Added missing highlight

* Fixed spelling of StackOverflowError

Co-authored-by: lcawl <lcawley@elastic.co>

* Brought back missing changelog for 87235 (#87370)

* Brought back missing changelog for 87235

* Regenerate release notes

* Regenerate release notes for BC3 (#87449)

* Regenerate release notes for BC3

* Re-applied manual fixes that Lisa Cawley used

* Fixed breaking changes generation

To match the manual edits done by Lisa Cawley

* Fixed failing test

Since the manual edits by Lisa removed the `-SNAPSHOT` from the docs
we remove it from the tests too.

* Remove changelogs for intermediate lucene upgrades

* Update release notes for BC4 (#87635)

* Update release notes for BC4

* Adding missing changelog for geo_grid

* Added missing highlight notes for 84250

* Update docs/reference/release-notes/highlights.asciidoc

Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com>

* Update docs/reference/release-notes/highlights.asciidoc

Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com>

* Backported fixes to original yaml

* Control sort order of release highlights

So that small changes to an individual highlight don't completely shuffle the entire document.

We also added links to the PRs from the highlight titles, for convenience.
Otherwise readers need to search the release notes for the changelog entry and click there,
which is a lot more work.

* Fixed failing test after new ordered highlights

Also made test verify re-ordering of highlights

Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com>

* Update release notes for BC5 (#87808)

* Update release notes for BC6 (#87912)

* Update release notes for BC9 (#88011)

* Regenerate release notes after version bump 8.4.0

And after forward porting 8.3.0 release notes changes.

* Regenerate after some changelog entries removed

* Re-prune the changelogs

* Removed three changelog entries

Co-authored-by: lcawl <lcawley@elastic.co>
Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com>

build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/release/BreakingChangesGenerator.java

@@ -74,6 +74,7 @@ public class BreakingChangesGenerator {
         bindings.put("deprecationsByNotabilityByArea", deprecationsByNotabilityByArea);
         bindings.put("isElasticsearchSnapshot", version.isSnapshot());
         bindings.put("majorDotMinor", version.major() + "." + version.minor());
+        bindings.put("majorDotMinorDotRevision", version.major() + "." + version.minor() + "." + version.revision());
         bindings.put("majorMinor", String.valueOf(version.major()) + version.minor());
         bindings.put("nextMajor", (version.major() + 1) + ".0");
         bindings.put("version", version);

build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/release/ChangelogEntry.java

@@ -99,6 +99,7 @@ public class ChangelogEntry {
 
     public void setHighlight(Highlight highlight) {
         this.highlight = highlight;
+        if (this.highlight != null) this.highlight.pr = this.pr;
     }
 
     public Breaking getBreaking() {
@@ -160,6 +161,7 @@ public class ChangelogEntry {
         private boolean notable;
         private String title;
         private String body;
+        private Integer pr;
 
         public boolean isNotable() {
             return notable;
@@ -189,6 +191,10 @@ public class ChangelogEntry {
             return generatedAnchor(this.title);
         }
 
+        public Integer getPr() {
+            return pr;
+        }
+
         @Override
         public boolean equals(Object o) {
             if (this == o) {

build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/release/ReleaseHighlightsGenerator.java

@@ -17,6 +17,7 @@ import java.io.FileWriter;
 import java.io.IOException;
 import java.nio.file.Files;
 import java.util.ArrayList;
+import java.util.Comparator;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
@@ -50,6 +51,7 @@ public class ReleaseHighlightsGenerator {
         final Map<Boolean, List<ChangelogEntry.Highlight>> groupedHighlights = entries.stream()
             .map(ChangelogEntry::getHighlight)
             .filter(Objects::nonNull)
+            .sorted(Comparator.comparingInt(ChangelogEntry.Highlight::getPr))
             .collect(Collectors.groupingBy(ChangelogEntry.Highlight::isNotable, Collectors.toList()));
 
         final List<ChangelogEntry.Highlight> notableHighlights = groupedHighlights.getOrDefault(true, List.of());

build-tools-internal/src/main/resources/templates/breaking-changes.asciidoc

@@ -9,14 +9,16 @@ your application to {es} ${majorDotMinor}.
 
 See also <<release-highlights>> and <<es-release-notes>>.
 <% if (isElasticsearchSnapshot) { %>
-coming::[${version}]
+coming::[${majorDotMinorDotRevision}]
 <% } %>
 
 [discrete]
 [[breaking-changes-${majorDotMinor}]]
 === Breaking changes
 <% if (breakingByNotabilityByArea.isEmpty()) { %>
+// tag::notable-breaking-changes[]
 There are no breaking changes in {es} ${majorDotMinor}.
+// end::notable-breaking-changes[]
 <% } else { %>
 The following changes in {es} ${majorDotMinor} might affect your applications
 and prevent them from operating normally.

build-tools-internal/src/main/resources/templates/release-highlights.asciidoc

@@ -32,7 +32,7 @@ if (notableHighlights.isEmpty()) { %>
 <% for (highlight in notableHighlights) { %>
 [discrete]
 [[${ highlight.anchor }]]
-=== ${highlight.title}
+=== {es-pull}${highlight.pr}[${highlight.title}]
 ${highlight.body.trim()}
 <% } %>
 // end::notable-highlights[]
@@ -40,6 +40,6 @@ ${highlight.body.trim()}
 <% for (highlight in nonNotableHighlights) { %>
 [discrete]
 [[${ highlight.anchor }]]
-=== ${highlight.title}
+=== {es-pull}${highlight.pr}[${highlight.title}]
 ${highlight.body.trim()}
 <% } %>

build-tools-internal/src/test/java/org/elasticsearch/gradle/internal/release/ReleaseHighlightsGeneratorTest.java

@@ -60,31 +60,24 @@ public class ReleaseHighlightsGeneratorTest {
     }
 
     private List<ChangelogEntry> getEntries() {
-        ChangelogEntry entry1 = new ChangelogEntry();
-        ChangelogEntry.Highlight highlight1 = new ChangelogEntry.Highlight();
-        entry1.setHighlight(highlight1);
-
-        highlight1.setNotable(true);
-        highlight1.setTitle("Notable release highlight number 1");
-        highlight1.setBody("Notable release body number 1");
-
-        ChangelogEntry entry2 = new ChangelogEntry();
-        ChangelogEntry.Highlight highlight2 = new ChangelogEntry.Highlight();
-        entry2.setHighlight(highlight2);
-
-        highlight2.setNotable(true);
-        highlight2.setTitle("Notable release highlight number 2");
-        highlight2.setBody("Notable release body number 2");
+        ChangelogEntry entry1 = makeChangelogEntry(1, true);
+        ChangelogEntry entry2 = makeChangelogEntry(2, true);
+        ChangelogEntry entry3 = makeChangelogEntry(3, false);
+        // Return unordered list, to test correct re-ordering
+        return List.of(entry2, entry1, entry3);
+    }
 
-        ChangelogEntry entry3 = new ChangelogEntry();
-        ChangelogEntry.Highlight highlight3 = new ChangelogEntry.Highlight();
-        entry3.setHighlight(highlight3);
+    private ChangelogEntry makeChangelogEntry(int pr, boolean notable) {
+        ChangelogEntry entry = new ChangelogEntry();
+        entry.setPr(pr);
+        ChangelogEntry.Highlight highlight = new ChangelogEntry.Highlight();
+        entry.setHighlight(highlight);
 
-        highlight3.setNotable(false);
-        highlight3.setTitle("Notable release highlight number 3");
-        highlight3.setBody("Notable release body number 3");
+        highlight.setNotable(notable);
+        highlight.setTitle("Notable release highlight number " + pr);
+        highlight.setBody("Notable release body number " + pr);
 
-        return List.of(entry1, entry2, entry3);
+        return entry;
     }
 
     private String getResource(String name) throws Exception {

build-tools-internal/src/test/resources/org/elasticsearch/gradle/internal/release/BreakingChangesGeneratorTest.generateMigrationFile.asciidoc

@@ -9,7 +9,7 @@ your application to {es} 8.4.
 
 See also <<release-highlights>> and <<es-release-notes>>.
 
-coming::[8.4.0-SNAPSHOT]
+coming::[8.4.0]
 
 
 [discrete]

build-tools-internal/src/test/resources/org/elasticsearch/gradle/internal/release/ReleaseHighlightsGeneratorTest.generateFile.asciidoc

@@ -21,12 +21,12 @@ Other versions:
 
 [discrete]
 [[notable_release_highlight_number_1]]
-=== Notable release highlight number 1
+=== {es-pull}1[Notable release highlight number 1]
 Notable release body number 1
 
 [discrete]
 [[notable_release_highlight_number_2]]
-=== Notable release highlight number 2
+=== {es-pull}2[Notable release highlight number 2]
 Notable release body number 2
 
 // end::notable-highlights[]
@@ -34,6 +34,6 @@ Notable release body number 2
 
 [discrete]
 [[notable_release_highlight_number_3]]
-=== Notable release highlight number 3
+=== {es-pull}3[Notable release highlight number 3]
 Notable release body number 3
 

docs/changelog/86227.yaml

@@ -1,5 +0,0 @@
-pr: 86227
-summary: Upgrade to Lucene 9.2 snapshot
-area: Search
-type: upgrade
-issues: []

docs/changelog/86852.yaml

@@ -1,5 +0,0 @@
-pr: 86852
-summary: Upgrade to lucene snapshot 978eef5459c
-area: Search
-type: upgrade
-issues: []

docs/reference/migration/index.asciidoc

@@ -1,9 +1,13 @@
 include::migration_intro.asciidoc[]
 
-* <<migrating-8.1,Migrating to 8.2>>
+* <<migrating-8.4,Migrating to 8.4>>
+* <<migrating-8.3,Migrating to 8.3>>
+* <<migrating-8.2,Migrating to 8.2>>
 * <<migrating-8.1,Migrating to 8.1>>
 * <<migrating-8.0,Migrating to 8.0>>
 
+include::migrate_8_4.asciidoc[]
+include::migrate_8_3.asciidoc[]
 include::migrate_8_2.asciidoc[]
 include::migrate_8_1.asciidoc[]
 include::migrate_8_0.asciidoc[]

docs/reference/migration/migrate_8_3.asciidoc

@@ -0,0 +1,64 @@
+[[migrating-8.3]]
+== Migrating to 8.3
+++++
+<titleabbrev>8.3</titleabbrev>
+++++
+
+This section discusses the changes that you need to be aware of when migrating
+your application to {es} 8.3.
+
+See also <<release-highlights>> and <<es-release-notes>>.
+
+coming::[8.3.0-SNAPSHOT]
+
+
+
+[discrete]
+[[breaking-changes-8.3]]
+=== Breaking changes
+
+There are no breaking changes in {es} 8.3.
+
+
+
+[discrete]
+[[deprecated-8.3]]
+=== Deprecations
+
+The following functionality has been deprecated in {es} 8.3
+and will be removed in a future version.
+While this won't have an immediate impact on your applications,
+we strongly encourage you take the described steps to update your code
+after upgrading to 8.3.
+
+To find out if you are using any deprecated functionality,
+enable <<deprecation-logging, deprecation logging>>.
+
+
+[discrete]
+[[deprecations_83_cluster_and_node_setting]]
+==== Cluster and node setting deprecations
+
+[[configuring_bind_dn_in_an_ldap_or_active_directory_ad_realm_without_corresponding_bind_password_deprecated]]
+.Configuring a bind DN in an LDAP or Active Directory (AD) realm without a corresponding bind password is deprecated
+[%collapsible]
+====
+*Details* +
+For LDAP or AD authentication realms, setting a bind DN (via the
+`xpack.security.authc.realms.ldap.*.bind_dn` realm setting) without a
+bind password is a misconfiguration that may prevent successful
+authentication to the node. In the next major release, nodes will fail
+to start if a bind DN is specified without a password.
+
+*Impact* +
+If you have a bind DN configured for an LDAP or AD authentication
+realm, set a bind password for {ref}/ldap-realm.html#ldap-realm-configuration[LDAP]
+or {ref}/active-directory-realm.html#ad-realm-configuration[Active Directory].
+Configuring a bind DN without a password generates a warning in the
+deprecation logs.
+
+*Note:* This deprecation only applies if your current LDAP or AD
+configuration specifies a bind DN without a password. This scenario is
+unlikely, but might impact a small subset of users.
+====
+

docs/reference/migration/migrate_8_4.asciidoc

@@ -0,0 +1,22 @@
+[[migrating-8.4]]
+== Migrating to 8.4
+++++
+<titleabbrev>8.4</titleabbrev>
+++++
+
+This section discusses the changes that you need to be aware of when migrating
+your application to {es} 8.4.
+
+See also <<release-highlights>> and <<es-release-notes>>.
+
+coming::[8.4.0]
+
+
+[discrete]
+[[breaking-changes-8.4]]
+=== Breaking changes
+
+// tag::notable-breaking-changes[]
+There are no breaking changes in {es} 8.4.
+// end::notable-breaking-changes[]
+

docs/reference/release-notes.asciidoc

@@ -6,6 +6,8 @@
 
 This section summarizes the changes in each release.
 
+* <<release-notes-8.4.0>>
+* <<release-notes-8.3.0>>
 * <<release-notes-8.2.3>>
 * <<release-notes-8.2.2>>
 * <<release-notes-8.2.1>>
@@ -24,6 +26,8 @@ This section summarizes the changes in each release.
 
 --
 
+include::release-notes/8.4.0.asciidoc[]
+include::release-notes/8.3.0.asciidoc[]
 include::release-notes/8.2.3.asciidoc[]
 include::release-notes/8.2.2.asciidoc[]
 include::release-notes/8.2.1.asciidoc[]

docs/reference/release-notes/8.3.0.asciidoc

@@ -0,0 +1,365 @@
+[[release-notes-8.3.0]]
+== {es} version 8.3.0
+
+coming[8.3.0]
+
+Also see <<breaking-changes-8.3,Breaking changes in 8.3>>.
+
+[[bug-8.3.0]]
+[float]
+=== Bug fixes
+
+Aggregations::
+* Allow `serial_diff` under `min_doc_count` aggs {es-pull}86401[#86401]
+* Allow bucket paths to specify `_count` within a bucket {es-pull}85720[#85720]
+* Fix a bug with flattened fields in terms aggregations {es-pull}87392[#87392]
+* Fix flaky `top_metrics` test {es-pull}86582[#86582] (issue: {es-issue}86377[#86377])
+* Fix: check field existence before trying to merge running stats {es-pull}86926[#86926]
+* Fix: ordering terms aggregation on top metrics null values {es-pull}85774[#85774]
+* Serialize interval in auto date histogram aggregation {es-pull}85473[#85473]
+
+Audit::
+* Fix audit logging to consistently include port number in `origin.address` {es-pull}86732[#86732]
+* Support removing ignore filters for audit logging {es-pull}87675[#87675] (issue: {es-issue}68588[#68588])
+
+Authentication::
+* An authorized user can disable a user with same name but different realm {es-pull}86473[#86473]
+* Fix clearing of `lastSuccessfulAuthCache` when clear all realm cache API is called {es-pull}86909[#86909] (issue: {es-issue}86650[#86650])
+
+Authorization::
+* Fix resolution of wildcard application privileges {es-pull}87293[#87293]
+
+CAT APIs::
+* Get hidden indices stats in `GET _cat/shards` {es-pull}86601[#86601] (issue: {es-issue}84656[#84656])
+
+CCR::
+* Prevent invalid datastream metadata when CCR follows a datastream with closed indices on the follower {es-pull}87076[#87076] (issue: {es-issue}87048[#87048])
+* Remove some blocking in CcrRepository {es-pull}87235[#87235]
+
+Cluster Coordination::
+* Add `master_timeout` support to voting config exclusions APIs {es-pull}86670[#86670]
+* Small fixes to clear voting config excls API {es-pull}87828[#87828]
+
+Discovery-Plugins::
+* [discovery-gce] Fix initialisation of transport in FIPS mode {es-pull}85817[#85817] (issue: {es-issue}85803[#85803])
+
+Distributed::
+* Enforce external id uniqueness during `DesiredNode` construction {es-pull}84227[#84227]
+
+Engine::
+* Fork to WRITE thread before failing shard in `updateCheckPoints` {es-pull}87458[#87458] (issue: {es-issue}87094[#87094])
+* Removing Blocking Wait for Close in `RecoverySourceHandler` {es-pull}86127[#86127] (issue: {es-issue}85839[#85839])
+
+Features::
+* Fix 'autoGeneratedTimestamp should not be set externally' error when retrying IndexRequest {es-pull}86184[#86184] (issue: {es-issue}83927[#83927])
+
+Geo::
+* Fix Geotile aggregations on `geo_shapes` for precision 0 {es-pull}87202[#87202] (issue: {es-issue}87201[#87201])
+* Fix `null_value` for array-valued `geo_point` fields {es-pull}85959[#85959]
+* Guard for adding null value tags to vector tiles {es-pull}87051[#87051]
+* Quantize geo queries to remove true negatives from search results {es-pull}85441[#85441] (issue: {es-issue}40891[#40891])
+
+Highlighting::
+* `FastVectorHighlighter` should use `ValueFetchers` to load source data {es-pull}85815[#85815] (issues: {es-issue}75011[#75011], {es-issue}84690[#84690], {es-issue}82458[#82458], {es-issue}80895[#80895])
+
+ILM+SLM::
+* Make the ILM Move to Error Step Batched {es-pull}85565[#85565] (issue: {es-issue}81880[#81880])
+
+Indices APIs::
+* Make `GetIndexAction` cancellable {es-pull}87681[#87681]
+
+Infra/Circuit Breakers::
+* Make CBE message creation more robust {es-pull}87881[#87881]
+
+Infra/Core::
+* Adjust osprobe assertion for burst cpu {es-pull}86990[#86990]
+* Clean up `DeflateCompressor` after exception {es-pull}87163[#87163] (issue: {es-issue}87160[#87160])
+* Error on direct creation of non-primary system index {es-pull}86707[#86707]
+* Fix null message in output {es-pull}86981[#86981]
+* Fix using `FilterOutputStream` without overriding bulk write {es-pull}86304[#86304]
+* Hide system indices and their aliases in upgraded clusters {es-pull}87125[#87125]
+* Refactor code to avoid JDK bug: JDK-8285835 {es-pull}86614[#86614]
+
+Infra/Logging::
+* Temporarily provide `SystemPropertiesPropertySource` {es-pull}87149[#87149]
+
+Infra/Node Lifecycle::
+* Upgrade folders after settings validation {es-pull}87319[#87319]
+
+Infra/Plugins::
+* Use Windows newlines when listing plugin information on Windows {es-pull}86408[#86408] (issue: {es-issue}86352[#86352])
+
+Infra/REST API::
+* Fix min node version before state recovery {es-pull}86482[#86482]
+
+Infra/Scripting::
+* Allow to sort by script value using `SemVer` semantics {es-pull}85990[#85990] (issues: {es-issue}85989[#85989], {es-issue}82287[#82287])
+* Script: Fix setter shortcut for unbridged setters {es-pull}86868[#86868]
+* Script: Load Whitelists as Resource {es-pull}87539[#87539]
+
+Infra/Settings::
+* Permit removal of archived index settings {es-pull}86107[#86107]
+
+Ingest::
+* Execute self-reference checks once per pipeline {es-pull}85926[#85926] (issue: {es-issue}85790[#85790])
+
+Java Low Level REST Client::
+* Do not retry client requests when failing with `ContentTooLargeException` {es-pull}87248[#87248] (issue: {es-issue}86041[#86041])
+
+License::
+* Consistent response for starting basic license {es-pull}86272[#86272] (issue: {es-issue}86244[#86244])
+
+Machine Learning::
+* Fix ML task auditor exception early in cluster lifecycle {es-pull}87023[#87023] (issue: {es-issue}87002[#87002])
+* Fix `WordPiece` tokenization of unknown words with known subwords {es-pull}87510[#87510]
+* Fix distribution change check for `change_point` aggregation {es-pull}86423[#86423]
+* Fixes inference timeout handling bug that throws unexpected `NullPointerException` {es-pull}87533[#87533]
+* Correct logic for restart from failover fine tuning hyperparameters for training classification and regression models {ml-pull}2251[#2251]
+* Fix possible source of "x = NaN, distribution = class   boost::math::normal_distribution<..." log errors training classification and regression models {ml-pull}2249[#2249]
+* Fix some bugs affecting decision to stop optimizing hyperparameters for training classification and regression models {ml-pull}2259[#2259]
+* Fix cause of "Must provide points at which to evaluate function" log error training classification and regression models {ml-pull}2268[#2268]
+* Fix a source of "Discarding sample = nan, weights = ..." log errors for time series anomaly detection {ml-pull}2286[#2286]
+
+Mapping::
+* Don't run `include_in_parent` when in `copy_to` context {es-pull}87123[#87123] (issue: {es-issue}87036[#87036])
+
+Network::
+* Reject `openConnection` attempt while closing {es-pull}86315[#86315] (issue: {es-issue}86249[#86249])
+
+Recovery::
+* Fail shard if STARTED after master failover {es-pull}87451[#87451] (issue: {es-issue}87367[#87367])
+
+SQL::
+* Fix FORMAT function to comply with Microsoft SQL Server specification {es-pull}86225[#86225] (issue: {es-issue}66560[#66560])
+* Implement binary format support for SQL clear cursor {es-pull}84230[#84230] (issue: {es-issue}53359[#53359])
+
+Search::
+* Add status field to Multi Search Template Responses {es-pull}85496[#85496] (issue: {es-issue}83029[#83029])
+* Fields API to allow fetching values when `_source` is disabled {es-pull}87267[#87267] (issue: {es-issue}87072[#87072])
+* Fix `_terms_enum` on unconfigured `constant_keyword` {es-pull}86191[#86191] (issues: {es-issue}86187[#86187], {es-issue}86267[#86267])
+* Fix status code when open point in time without `keep_alive` {es-pull}87011[#87011] (issue: {es-issue}87003[#87003])
+* Handle empty point values in `DiskUsage` API {es-pull}87826[#87826] (issue: {es-issue}87761[#87761])
+* Make sure to rewrite explain query on coordinator {es-pull}87013[#87013] (issue: {es-issue}64281[#64281])
+
+Security::
+* Make user and role name constraint consistent with max document ID {es-pull}86728[#86728] (issue: {es-issue}66020[#66020])
+* Security plugin close releasable realms {es-pull}87429[#87429] (issue: {es-issue}86286[#86286])
+
+Snapshot/Restore::
+* DONE should mean fully processed in snapshot status {es-pull}86414[#86414]
+* Distinguish missing and invalid repositories {es-pull}85551[#85551] (issue: {es-issue}85550[#85550])
+* Fork after calling `getRepositoryData` from `StoreRecovery` {es-pull}87264[#87264] (issue: {es-issue}87237[#87237])
+* Fork after calling `getRepositoryData` from `StoreRecovery` {es-pull}87254[#87254] (issue: {es-issue}87237[#87237])
+* Throw exception on illegal `RepositoryData` updates {es-pull}87654[#87654]
+* Upgrade Azure SDK to 12.16.0 {es-pull}86135[#86135]
+
+Stats::
+* Run `TransportClusterInfoActions` on MANAGEMENT pool {es-pull}87679[#87679]
+
+TSDB::
+* TSDB: fix the time_series in order collect priority {es-pull}85526[#85526]
+* TSDB: fix wrong initial value of tsidOrd in TimeSeriesIndexSearcher {es-pull}85713[#85713] (issue: {es-issue}85711[#85711])
+
+Transform::
+* Fix transform `_start` permissions to use stored headers in the config {es-pull}86802[#86802]
+* [Transforms] fix bug when unsetting retention policy {es-pull}87711[#87711]
+
+[[deprecation-8.3.0]]
+[float]
+=== Deprecations
+
+Authentication::
+* Configuring a bind DN in an LDAP or Active Directory (AD) realm without a corresponding bind password is deprecated {es-pull}85326[#85326] (issue: {es-issue}47191[#47191])
+
+[[enhancement-8.3.0]]
+[float]
+=== Enhancements
+
+Aggregations::
+* Improve min and max performance while in a `random_sampler` aggregation {es-pull}85118[#85118]
+
+Authentication::
+* Support configurable claims in JWT Realm Tokens {es-pull}86533[#86533]
+* Warn on user roles disabled due to licensing requirements for document or field level security {es-pull}85393[#85393] (issue: {es-issue}79207[#79207])
+* `TokenService` decode JWTs, change warn to debug {es-pull}86498[#86498]
+
+Authorization::
+* Add delete privilege to `kibana_system` for Synthetics {es-pull}85844[#85844]
+* Authorize painless execute as index action when an index is specified {es-pull}85512[#85512] (issue: {es-issue}86428[#86428])
+* Better error message for run-as denials {es-pull}85501[#85501] (issue: {es-issue}72904[#72904])
+* Improve "Has Privilege" performance for boolean-only response {es-pull}86685[#86685]
+* Relax restrictions for role names in roles API {es-pull}86604[#86604] (issue: {es-issue}86480[#86480])
+* [Osquery] Extend `kibana_system` role with an access to osquery_manager… {es-pull}86609[#86609]
+
+Autoscaling::
+* Add support for CPU ranges in desired nodes {es-pull}86434[#86434]
+
+Cluster Coordination::
+* Block joins while applier is busy {es-pull}84919[#84919]
+* Compute master task batch summary lazily {es-pull}86210[#86210]
+* Log `cluster.initial_master_nodes` at startup {es-pull}86101[#86101]
+* Reduce resource needs of join validation {es-pull}85380[#85380] (issue: {es-issue}83204[#83204])
+* Report pending joins in `ClusterFormationFailureHelper` {es-pull}85635[#85635]
+* Speed up map diffing (2) {es-pull}86375[#86375]
+
+Discovery-Plugins::
+* Remove redundant jackson dependencies from discovery-azure {es-pull}87898[#87898]
+
+Distributed::
+* Keep track of desired nodes cluster membership {es-pull}84165[#84165]
+
+Engine::
+* Cache immutable translog lastModifiedTime {es-pull}82721[#82721] (issue: {es-issue}82720[#82720])
+* Increase `force_merge` threadpool size based on the allocated processors {es-pull}87082[#87082] (issue: {es-issue}84943[#84943])
+* More optimal forced merges when max_num_segments is greater than 1 {es-pull}85065[#85065]
+
+Geo::
+* Support 'GeoJSON' in CartesianPoint for 'point' {es-pull}85442[#85442]
+* Support geo label position as runtime field {es-pull}86154[#86154]
+* Support geo label position through REST vector tiles API {es-pull}86458[#86458] (issue: {es-issue}86044[#86044])
+
+Health::
+* Add a basic check for tier preference and allocation filter clashing {es-pull}85071[#85071]
+* Add preflight checks to Health API to ensure health is obtainable {es-pull}86404[#86404]
+* Add tier information on health api migrate tiers user actions {es-pull}87486[#87486]
+* Health api add indicator doc links {es-pull}86904[#86904] (issue: {es-issue}86892[#86892])
+* Health api copy editing {es-pull}87010[#87010]
+* Return a default user action if no actions could be determined {es-pull}87079[#87079]
+
+ILM+SLM::
+* Make the ILM and SLM `history_index_enabled` settings dynamic {es-pull}86493[#86493]
+
+Indices APIs::
+* Batch execute template and pipeline cluster state operations {es-pull}86017[#86017]
+
+Infra/Core::
+* Add mapping for tags for the elastic agent {es-pull}86298[#86298]
+* Expand jar hell to include modules {es-pull}86622[#86622]
+* Faster GET _cluster/settings API {es-pull}86405[#86405] (issue: {es-issue}82342[#82342])
+* Faster string writes by saving stream flushes {es-pull}86114[#86114]
+* Fleet: Add `start_time` and `minimum_execution_duration` attributes to actions {es-pull}86167[#86167]
+* Force property expansion for security policy {es-pull}87396[#87396]
+* Refactor array part into a `BytesRefArray` which can be serialized and … {es-pull}85826[#85826]
+* Speed up ip v4 parser {es-pull}86253[#86253]
+* Use varhandles for primitive type conversion in more places {es-pull}85577[#85577] (issue: {es-issue}78823[#78823])
+
+Infra/Scripting::
+* Script: add ability to alias classes in whitelist {es-pull}86899[#86899]
+
+Ingest::
+* Iteratively execute synchronous ingest processors {es-pull}84250[#84250] (issue: {es-issue}84274[#84274])
+* Skip `ensureNoSelfReferences` check in `IngestService` {es-pull}87337[#87337]
+
+License::
+* Initialize active realms without logging a message {es-pull}86134[#86134] (issue: {es-issue}81380[#81380])
+
+Machine Learning::
+* A text categorization aggregation that works like ML categorization {es-pull}80867[#80867]
+* Add new _infer endpoint for all supervised models and deprecate deployment infer api {es-pull}86361[#86361]
+* Adds new `question_answering` NLP task for extracting answers to questions from a document {es-pull}85958[#85958]
+* Adds start and end params to `_preview` and excludes cold/frozen tiers from unbounded previews {es-pull}86989[#86989]
+* Adjust automatic JVM heap sizing for dedicated ML nodes {es-pull}86399[#86399]
+* Replace the implementation of the `categorize_text` aggregation {es-pull}85872[#85872]
+* Upgrade PyTorch to version 1.11 {ml-pull}2233[#2233], {ml-pull}2235[#2235],{ml-pull}2238[#2238]
+* Upgrade zlib to version 1.2.12 on Windows {ml-pull}2253[#2253]
+* Upgrade libxml2 to version 2.9.14 on Linux and Windows {ml-pull}2287[#2287]
+* Improve time series model stability and anomaly scoring consistency for data
+  for which many buckets are empty {ml-pull}2267[#2267]
+* Address root cause for actual equals typical equals zero anomalies {ml-pull}2270[#2270]
+* Better handling of outliers in update immediately after detecting changes in time series {ml-pull}2280[#2280]
+
+Mapping::
+* Intern field names in Mappers {es-pull}86301[#86301]
+* Replace BYTE_BLOCK_SIZE - 2 with indexWriter#MAX_TERM_LENGTH {es-pull}85518[#85518]
+
+Network::
+* Log node identity at startup {es-pull}85773[#85773]
+
+Search::
+* GeoBoundingBox query should work on bounding box with equal latitude or longitude {es-pull}85788[#85788] (issue: {es-issue}77717[#77717])
+* Improve error message for search API url parameters {es-pull}86984[#86984] (issue: {es-issue}79719[#79719])
+
+Security::
+* Add run-as support for OAuth2 tokens {es-pull}86680[#86680]
+* Relax username restrictions for User APIs {es-pull}86398[#86398] (issue: {es-issue}86326[#86326])
+* User Profile - Add hint support to SuggestProfiles API {es-pull}85890[#85890]
+* User Profile - Add new action origin and internal user {es-pull}86026[#86026]
+* User Profile - Support request cancellation on HTTP disconnect {es-pull}86332[#86332]
+* User Profile - add caching for `hasPrivileges` check {es-pull}86543[#86543]
+
+Snapshot/Restore::
+* Add parameter to exclude indices in a snapshot from response {es-pull}86269[#86269] (issue: {es-issue}82937[#82937])
+
+Stats::
+* Add documentation for "io_time_in_millis" {es-pull}84911[#84911]
+
+TLS::
+* Set `serverAuth` extended key usage for generated certificates and CSRs {es-pull}86311[#86311] (issue: {es-issue}81067[#81067])
+
+TSDB::
+* Aggregation Execution Context add timestamp provider {es-pull}85850[#85850]
+
+Transform::
+* Prefer secondary auth headers for transforms {es-pull}86757[#86757]
+* Support `range` aggregation in transform {es-pull}86501[#86501]
+
+[[feature-8.3.0]]
+[float]
+=== New features
+
+Authorization::
+* Has privileges API for profiles {es-pull}85898[#85898]
+
+Geo::
+* New geo_grid query to be used with geogrid aggregations {es-pull}86596[#86596] (issue: {es-issue}85727[#85727])
+
+Health::
+* Add support for `impact_areas` to health impacts {es-pull}85830[#85830] (issue: {es-issue}85829[#85829])
+* Add troubleshooting guides to shards allocation actions {es-pull}87078[#87078]
+* Adding potential impacts to remaining health indicators {es-pull}86197[#86197]
+* Health api drill down {es-pull}85234[#85234] (issue: {es-issue}84793[#84793])
+* New service to keep track of the master history as seen from each node {es-pull}85941[#85941]
+* Sorting impact index names by index priority {es-pull}85347[#85347]
+
+Mapping::
+* Add support for dots in field names for metrics usecases {es-pull}86166[#86166] (issue: {es-issue}63530[#63530])
+* Synthetic source {es-pull}85649[#85649]
+
+SQL::
+* SQ: Allow partial results in SQL queries {es-pull}85897[#85897] (issue: {es-issue}33148[#33148])
+
+Search::
+* Snapshots as simple archives {es-pull}86261[#86261] (issue: {es-issue}81210[#81210])
+
+TSDB::
+* TSDB: Implement downsampling on time-series indices {es-pull}85708[#85708] (issues: {es-issue}69799[#69799], {es-issue}65769[#65769])
+
+[[upgrade-8.3.0]]
+[float]
+=== Upgrades
+
+Infra/CLI::
+* Upgrade procrun executables to 1.3.1 {es-pull}86710[#86710]
+
+Infra/Core::
+* Upgrade jackson to 2.13.2 {es-pull}86051[#86051]
+
+Ingest::
+* Upgrading to tika 2.4 {es-pull}86015[#86015]
+
+Network::
+* Upgrade to Netty 4.1.76 {es-pull}86252[#86252]
+
+Packaging::
+* Update Iron Bank base image to 8.6 {es-pull}86796[#86796]
+
+SQL::
+* Update dependency - JLine - to v 3.21.0 {es-pull}83767[#83767] (issue: {es-issue}83575[#83575])
+
+Search::
+* Update to public lucene 9.2.0 release {es-pull}87162[#87162]
+
+Snapshot/Restore::
+* Upgrade GCS Plugin to 1.118.1 {es-pull}87800[#87800]
+
+

docs/reference/release-notes/8.4.0.asciidoc

@@ -0,0 +1,181 @@
+[[release-notes-8.4.0]]
+== {es} version 8.4.0
+
+coming[8.4.0]
+
+Also see <<breaking-changes-8.4,Breaking changes in 8.4>>.
+
+[[bug-8.4.0]]
+[float]
+=== Bug fixes
+
+Aggregations::
+* Make the metric in the `buckets_path` parameter optional {es-pull}87220[#87220] (issue: {es-issue}72983[#72983])
+
+Allocation::
+* Clamp auto-expand replicas to the closest value {es-pull}87505[#87505] (issue: {es-issue}84788[#84788])
+
+Authentication::
+* Fix unique realm name check to cover default realms {es-pull}87999[#87999]
+
+Authorization::
+* Add rollover permissions for `remote_monitoring_agent` {es-pull}87717[#87717] (issue: {es-issue}84161[#84161])
+
+Autoscaling::
+* Do not include desired nodes in snapshots {es-pull}87695[#87695]
+
+EQL::
+* Avoid attempting PIT close on PIT open failure {es-pull}87498[#87498]
+
+Health::
+* Using the correct connection to fetch remote master history {es-pull}87299[#87299]
+
+Highlighting::
+* Handle ordering in plain highlighter for multiple inputs {es-pull}87414[#87414] (issue: {es-issue}87210[#87210])
+
+ILM+SLM::
+* Batch ILM move to retry step task update {es-pull}86759[#86759]
+
+Infra/Core::
+* Disallow three-digit minor and revision versions {es-pull}87338[#87338]
+
+Ingest::
+* Don't ignore pipeline for upserts in bulk api {es-pull}87719[#87719] (issue: {es-issue}87131[#87131])
+* Geoip processor should respect the `ignore_missing` in case of missing database {es-pull}87793[#87793] (issue: {es-issue}87345[#87345])
+
+Machine Learning::
+* Improve trained model stats API performance {es-pull}87978[#87978]
+
+SQL::
+* Fix date range checks {es-pull}87151[#87151] (issue: {es-issue}77179[#77179])
+
+Snapshot/Restore::
+* Use the provided SAS token without SDK sanitation that can produce invalid signatures {es-pull}88155[#88155] (issue: {es-issue}88140[#88140])
+
+Transform::
+* Execute `_refresh` separately from DBQ, with system permissions {es-pull}88005[#88005] (issue: {es-issue}88001[#88001])
+
+[[enhancement-8.4.0]]
+[float]
+=== Enhancements
+
+Aggregations::
+* Minor `RangeAgg` optimization {es-pull}86935[#86935] (issue: {es-issue}84262[#84262])
+* Speed counting filters/range/date_histogram aggs {es-pull}81322[#81322]
+* Update bucket metric pipeline agg paths to allow intermediate single bucket and bucket qualified multi-bucket aggs {es-pull}85729[#85729]
+
+Allocation::
+* Add debug information to `ReactiveReason` about assigned and unassigned shards {es-pull}86132[#86132] (issue: {es-issue}85243[#85243])
+* Use desired nodes during data tier allocation decisions {es-pull}87735[#87735]
+
+Audit::
+* User Profile - audit support for security domain {es-pull}87097[#87097]
+
+Authorization::
+* App permissions with action patterns do not retrieve privileges {es-pull}85455[#85455]
+* Cancellable Profile Has Privilege check {es-pull}87224[#87224]
+* Return action denied error when user with insufficient privileges (`manage_own_api_key`) attempts a grant API key request {es-pull}87461[#87461] (issue: {es-issue}87438[#87438])
+
+Autoscaling::
+* Add processors to autoscaling capacity response {es-pull}87895[#87895]
+* Keep track of desired nodes status in cluster state {es-pull}87474[#87474]
+
+Cluster Coordination::
+* Expose segment details in PCSS debug log {es-pull}87412[#87412]
+* Report overall mapping size in cluster stats {es-pull}87556[#87556]
+
+Data streams::
+* Give doc-value-only mappings to numeric fields on metrics templates {es-pull}87100[#87100]
+
+Distributed::
+* Make Desired Nodes API operator-only {es-pull}87778[#87778] (issue: {es-issue}87777[#87777])
+
+FIPS::
+* Log warning when hash function used by cache is not recommended in FIPS mode {es-pull}86740[#86740]
+* Log warning when hashers for stored API keys or service tokens are not compliant with FIPS {es-pull}87363[#87363]
+
+Geo::
+* Optimize geogrid aggregations for singleton points {es-pull}87439[#87439]
+* Use a faster but less accurate log algorithm for computing Geotile Y coordinate {es-pull}87515[#87515]
+
+Health::
+* Adding a transport action to get cluster formation info {es-pull}87306[#87306]
+* Adding additional capability to the `master_is_stable` health indicator service {es-pull}87482[#87482]
+* Creating a transport action for the `CoordinationDiagnosticsService` {es-pull}87984[#87984]
+* Move the master stability logic into its own service separate from the `HealthIndicatorService` {es-pull}87672[#87672]
+* Remove cluster block preflight check from health api {es-pull}87520[#87520] (issue: {es-issue}87464[#87464])
+
+Infra/Core::
+* Improve console exception messages {es-pull}87942[#87942]
+* Stop making index read-only when executing force merge index lifecycle management action {es-pull}81162[#81162] (issue: {es-issue}81162[#81162])
+* Stream input and output support for optional collections {es-pull}88127[#88127]
+* Update version of internal http client {es-pull}87491[#87491]
+
+Infra/Logging::
+* Catch an exception when formatting a string fails {es-pull}87132[#87132]
+
+Ingest::
+* Allow pipeline processor to ignore missing pipelines {es-pull}87354[#87354]
+* Move the ingest attachment processor to the default distribution {es-pull}87989[#87989]
+* Only perform `ensureNoSelfReferences` check during ingest when needed {es-pull}87352[#87352] (issue: {es-issue}87335[#87335])
+* Removing `BouncyCastle` dependencies from ingest-attachment plugin {es-pull}88031[#88031]
+
+Machine Learning::
+* Add authorization info to ML config listings {es-pull}87884[#87884]
+* Expand allowed NER labels to be any I-O-B tagged labels {es-pull}87091[#87091]
+* Improve scalability of NLP models {es-pull}87366[#87366]
+
+Mapping::
+* Speed up `NumberFieldMapper` {es-pull}85688[#85688]
+
+Monitoring::
+* JvmService use SingleObjectCache {es-pull}87236[#87236]
+
+Network::
+* Allow start cluster with unreachable remote clusters {es-pull}87298[#87298]
+
+Performance::
+* Warn about impact of large readahead on search {es-pull}88007[#88007]
+
+Query Languages::
+* Add support for VERSION field type in SQL and EQL {es-pull}87590[#87590] (issue: {es-issue}83375[#83375])
+
+Rollup::
+* [TSDB] Add Kahan support to downsampling summation {es-pull}87554[#87554]
+
+SQL::
+* Implement support for partial search results in SQL CLI {es-pull}86982[#86982] (issue: {es-issue}86082[#86082])
+
+Search::
+* Add mapping stats for indexed `dense_vectors` {es-pull}86859[#86859]
+
+Security::
+* Automatically close idle connections in OIDC back-channel {es-pull}87773[#87773]
+* Support exists query for API key query {es-pull}87229[#87229]
+
+Snapshot/Restore::
+* Make snapshot deletes not block the repository during data blob deletes {es-pull}86514[#86514]
+* Update HDFS Repository to HDFS 3.3.3 {es-pull}88039[#88039]
+
+Transform::
+* Add authorization info to transform config listings {es-pull}87570[#87570]
+* Implement per-transform num_failure_retries setting {es-pull}87361[#87361]
+
+[[feature-8.4.0]]
+[float]
+=== New features
+
+Health::
+* Master stability health indicator part 1 (when a master has been seen recently) {es-pull}86524[#86524]
+
+Infra/Logging::
+* Stable logging API - the basic use case {es-pull}86612[#86612]
+
+[[upgrade-8.4.0]]
+[float]
+=== Upgrades
+
+Network::
+* Upgrade to Netty 4.1.77 {es-pull}86630[#86630]
+
+

docs/reference/release-notes/highlights.asciidoc

@@ -1,6 +1,8 @@
 [[release-highlights]]
 == What's new in {minor-version}
 
+coming::[{minor-version}]
+
 Here are the highlights of what's new and improved in {es} {minor-version}!
 ifeval::[\{release-state}\"!=\"unreleased\"]
 For detailed information about this release, see the <<es-release-notes>> and
@@ -10,7 +12,8 @@ endif::[]
 // Add previous release to the list
 Other versions:
 
-{ref-bare}/8.2/release-highlights.html[8.2]
+{ref-bare}/8.3/release-highlights.html[8.3]
+| {ref-bare}/8.2/release-highlights.html[8.2]
 | {ref-bare}/8.1/release-highlights.html[8.1]
 | {ref-bare}/8.0/release-highlights.html[8.0]
 
@@ -24,8 +27,3 @@ Other versions:
 // end::notable-highlights[]
 
 
-[discrete]
-[[integrate_filtering_support_for_approximate_nearest_neighbor_search]]
-=== Integrate filtering support for approximate nearest neighbor search
-The {ref}/knn-search-api.html[_knn_search endpoint] now has a "filter" option that allows to return only the nearest documents that satisfy the provided filter
-