[[release-notes-8.14.0]] == {es} version 8.14.0 Also see <>. [[breaking-8.14.0]] [float] === Breaking changes Security:: * Prevent DLS/FLS if `replication` is assigned {es-pull}108600[#108600] * Apply stricter Document Level Security (DLS) rules for the validate query API with the rewrite parameter {es-pull}105709[#105709] * Apply stricter Document Level Security (DLS) rules for terms aggregations when min_doc_count is set to 0 {es-pull}105714[#105714] [[bug-8.14.0]] [float] === Bug fixes Aggregations:: * Cross check livedocs for terms aggs when index access control list is non-null {es-pull}105714[#105714] * ESQL: Enable VALUES agg for datetime {es-pull}107016[#107016] * Fix IOOBE in TTest aggregation when using filters {es-pull}109034[#109034] * Validate stats formatting in standard `InternalStats` constructor {es-pull}107678[#107678] (issue: {es-issue}107671[#107671]) Application:: * [Bugfix] Connector API - fix status serialisation issue in termquery {es-pull}108365[#108365] * [Connector API] Fix bug with filtering validation toXContent {es-pull}107467[#107467] * [Connector API] Fix bug with parsing *_doc_count nullable fields {es-pull}108854[#108854] * [Connector API] Fix bug with with wrong target index for access control sync {es-pull}109097[#109097] Authorization:: * Users with monitor privileges can access async_search/status endpoint even when setting keep_alive {es-pull}107383[#107383] CAT APIs:: * Fix numeric sorts in `_cat/nodes` {es-pull}106189[#106189] (issue: {es-issue}48070[#48070]) CCR:: * Add ?master_timeout query parameter to ccr apis {es-pull}105168[#105168] CRUD:: * Fix `noop_update_total` is not being updated when using the `_bulk` {es-pull}105745[#105745] (issue: {es-issue}105742[#105742]) * Use correct system index bulk executor {es-pull}106150[#106150] Cluster Coordination:: * Fix support for infinite `?master_timeout` {es-pull}107050[#107050] Data streams:: * Add non-indexed fields to ecs templates {es-pull}106714[#106714] * Fix bulk NPE when retrying failure redirect after cluster block {es-pull}107598[#107598] * Improve error message when rolling over DS alias {es-pull}106708[#106708] (issue: {es-issue}106137[#106137]) * Only skip deleting a downsampled index if downsampling is in progress as part of DSL retention {es-pull}109020[#109020] Downsampling:: * Fix downsample action request serialization {es-pull}106919[#106919] (issue: {es-issue}106917[#106917]) EQL:: * Use #addWithoutBreaking when adding a negative number of bytes to the circuit breaker in `SequenceMatcher` {es-pull}107655[#107655] ES|QL:: * ESQL: Allow reusing BUCKET grouping expressions in aggs {es-pull}107578[#107578] * ESQL: Disable quoting in FROM command {es-pull}108431[#108431] * ESQL: Fix MV_DEDUPE when using data from an index {es-pull}107577[#107577] (issue: {es-issue}104745[#104745]) * ESQL: Fix error message when failing to resolve aggregate groupings {es-pull}108101[#108101] (issue: {es-issue}108053[#108053]) * ESQL: Fix treating all fields as MV in COUNT pushdown {es-pull}106720[#106720] * ESQL: Re-enable logical dependency check {es-pull}105860[#105860] * ESQL: median, count and `count_distinct` over constants {es-pull}107414[#107414] (issues: {es-issue}105248[#105248], {es-issue}104900[#104900]) * ES|QL fix no-length substring with supplementary (4-byte) character {es-pull}107183[#107183] * ES|QL: Fix usage of IN operator with TEXT fields {es-pull}106654[#106654] (issue: {es-issue}105379[#105379]) * ES|QL: Improve support for TEXT fields in functions {es-pull}106810[#106810] * Fix docs generation of signatures for variadic functions {es-pull}107865[#107865] * [ESQL] Mark `date_diff` as requiring all three arguments {es-pull}108834[#108834] (issue: {es-issue}108383[#108383]) Health:: * Don't stop checking if the `HealthNode` persistent task is present {es-pull}105449[#105449] (issue: {es-issue}98926[#98926]) * Health monitor concurrency fixes {es-pull}105674[#105674] (issue: {es-issue}105065[#105065]) Highlighting:: * Check preTags and postTags params for empty values {es-pull}106396[#106396] (issue: {es-issue}69009[#69009]) * added fix for inconsistent text trimming in Unified Highlighter {es-pull}99961[#99961] (issue: {es-issue}101803[#101803]) Infra/CLI:: * Workaround G1 bug for JDK 22 and 22.0.1 {es-pull}108571[#108571] Infra/Core:: * Add a check for the same feature being declared regular and historical {es-pull}106285[#106285] * Fix `AffixSetting.exists` to include secure settings {es-pull}106745[#106745] * Fix regression in get index settings (human=true) where the version was not displayed in human-readable format {es-pull}107447[#107447] * Nativeaccess: try to load all located libsystemds {es-pull}108238[#108238] (issue: {es-issue}107878[#107878]) * Update several references to `IndexVersion.toString` to use `toReleaseVersion` {es-pull}107828[#107828] (issue: {es-issue}107821[#107821]) * Update several references to `TransportVersion.toString` to use `toReleaseVersion` {es-pull}107902[#107902] Infra/Logging:: * Log when update AffixSetting using addAffixMapUpdateConsumer {es-pull}97072[#97072] Infra/Node Lifecycle:: * Consider `ShardRouting` roles when calculating shard copies in shutdown status {es-pull}106063[#106063] * Wait indefintely for http connections on shutdown by default {es-pull}106511[#106511] Infra/Scripting:: * Guard against a null scorer in painless execute {es-pull}109048[#109048] (issue: {es-issue}43541[#43541]) * Painless: Apply true regex limit factor with FIND and MATCH operation {es-pull}105670[#105670] Ingest Node:: * Catching `StackOverflowErrors` from bad regexes in `GsubProcessor` {es-pull}106851[#106851] * Fix `uri_parts` processor behaviour for missing extensions {es-pull}105689[#105689] (issue: {es-issue}105612[#105612]) * Remove leading is_ prefix from Enterprise geoip docs {es-pull}108518[#108518] * Slightly better geoip `databaseType` validation {es-pull}106889[#106889] License:: * Fix lingering license warning header {es-pull}108031[#108031] (issue: {es-issue}107573[#107573]) Machine Learning:: * Fix NPE in ML assignment notifier {es-pull}107312[#107312] * Fix `startOffset` must be non-negative error in XLMRoBERTa tokenizer {es-pull}107891[#107891] (issue: {es-issue}104626[#104626]) * Fix the position of spike, dip and distribution changes bucket when the sibling aggregation includes empty buckets {es-pull}106472[#106472] * Make OpenAI embeddings parser more flexible {es-pull}106808[#106808] Mapping:: * Dedupe terms in terms queries {es-pull}106381[#106381] * Extend support of `allowedFields` to `getMatchingFieldNames` and `getAllFields` {es-pull}106862[#106862] * Fix for raw mapping merge of fields named "properties" {es-pull}108867[#108867] (issue: {es-issue}108866[#108866]) * Handle infinity during synthetic source construction for scaled float field {es-pull}107494[#107494] (issue: {es-issue}107101[#107101]) * Handle pass-through subfields with deep nesting {es-pull}106767[#106767] * Wrap "Pattern too complex" exception into an `IllegalArgumentException` {es-pull}109173[#109173] Network:: * Fix HTTP corner-case response leaks {es-pull}105617[#105617] Search:: * Add `internalClusterTest` for and fix leak in `ExpandSearchPhase` {es-pull}108562[#108562] (issue: {es-issue}108369[#108369]) * Avoid attempting to load the same empty field twice in fetch phase {es-pull}107551[#107551] * Bugfix: Disable eager loading `BitSetFilterCache` on Indexing Nodes {es-pull}105791[#105791] * Cross-cluster painless/execute actions should check permissions only on target remote cluster {es-pull}105360[#105360] * Fix error 500 on invalid `ParentIdQuery` {es-pull}105693[#105693] (issue: {es-issue}105366[#105366]) * Fix range queries for float/half_float fields when bounds are out of type's range {es-pull}106691[#106691] * Fixing NPE when requesting [_none_] for `stored_fields` {es-pull}104711[#104711] * Fork when handling remote field-caps responses {es-pull}107370[#107370] * Handle parallel calls to `createWeight` when profiling is on {es-pull}108041[#108041] (issues: {es-issue}104131[#104131], {es-issue}104235[#104235]) * Harden field-caps request dispatcher {es-pull}108736[#108736] * Replace `UnsupportedOperationException` with `IllegalArgumentException` for non-existing columns {es-pull}107038[#107038] * Unable to retrieve multiple stored field values {es-pull}106575[#106575] * Validate `model_id` is required when using the `learning_to_rank` rescorer {es-pull}107743[#107743] Security:: * Disable validate when rewrite parameter is sent and the index access control list is non-null {es-pull}105709[#105709] * Fix field caps and field level security {es-pull}106731[#106731] Snapshot/Restore:: * Fix double-pausing shard snapshot {es-pull}109148[#109148] (issue: {es-issue}109143[#109143]) * Treat 404 as empty register in `AzureBlobStore` {es-pull}108900[#108900] (issue: {es-issue}108504[#108504]) * `SharedBlobCacheService.maybeFetchRegion` should use `computeCacheFileRegionSize` {es-pull}106685[#106685] TSDB:: * Flip dynamic mapping condition when create tsid {es-pull}105636[#105636] Transform:: * Consolidate permissions checks {es-pull}106413[#106413] (issue: {es-issue}105794[#105794]) * Disable PIT for remote clusters {es-pull}107969[#107969] * Make force-stopping the transform always remove persistent task from cluster state {es-pull}106989[#106989] (issue: {es-issue}106811[#106811]) * Only trigger action once per thread {es-pull}107232[#107232] (issue: {es-issue}107215[#107215]) * [Transform] Auto retry Transform start {es-pull}106243[#106243] Vector Search:: * Fix multithreading copies in lib vec {es-pull}108802[#108802] * [8.14] Fix multithreading copies in lib vec {es-pull}108810[#108810] [[deprecation-8.14.0]] [float] === Deprecations Mapping:: * Deprecate allowing `fields` in scenarios where it is ignored {es-pull}106031[#106031] [[enhancement-8.14.0]] [float] === Enhancements Aggregations:: * Add a `PriorityQueue` backed by `BigArrays` {es-pull}106361[#106361] * All new `shard_seed` parameter for `random_sampler` agg {es-pull}104830[#104830] Allocation:: * Add allocation stats {es-pull}105894[#105894] * Add index forecasts to /_cat/allocation output {es-pull}97561[#97561] Application:: * [Profiling] Add TopN Functions API {es-pull}106860[#106860] * [Profiling] Allow to override index settings {es-pull}106172[#106172] * [Profiling] Speed up serialization of flamegraph {es-pull}105779[#105779] Authentication:: * Support Profile Activate with JWTs with client authn {es-pull}105439[#105439] (issue: {es-issue}105342[#105342]) Authorization:: * Allow users to get status of own async search tasks {es-pull}106638[#106638] * [Security Solution] Add `read` permission for third party agent indices for `kibana_system` {es-pull}107046[#107046] Data streams:: * Add data stream lifecycle to kibana reporting template {es-pull}106259[#106259] ES|QL:: * Add ES|QL Locate function {es-pull}106899[#106899] (issue: {es-issue}106818[#106818]) * Add ES|QL signum function {es-pull}106866[#106866] * Add status for enrich operator {es-pull}106036[#106036] * Add two new OGC functions ST_X and ST_Y {es-pull}105768[#105768] * Adjust array resizing in block builder {es-pull}106934[#106934] * Bulk loading enrich fields in ESQL {es-pull}106796[#106796] * ENRICH support for TEXT fields {es-pull}106435[#106435] (issue: {es-issue}105384[#105384]) * ESQL: Add timers to many status results {es-pull}105421[#105421] * ESQL: Allow grouping key inside stats expressions {es-pull}106579[#106579] * ESQL: Introduce expression validation phase {es-pull}105477[#105477] (issue: {es-issue}105425[#105425]) * ESQL: Log queries at debug level {es-pull}108257[#108257] * ESQL: Regex improvements {es-pull}106429[#106429] * ESQL: Sum of constants {es-pull}105454[#105454] * ESQL: Support ST_DISJOINT {es-pull}107007[#107007] * ESQL: Support partially folding CASE {es-pull}106094[#106094] * ESQL: Use faster field caps {es-pull}105067[#105067] * ESQL: extend BUCKET with spans {es-pull}107272[#107272] * ESQL: perform a reduction on the data node {es-pull}106516[#106516] * Expand support for ENRICH to full set supported by ES ingest processors {es-pull}106186[#106186] (issue: {es-issue}106162[#106162]) * Introduce ordinal bytesref block {es-pull}106852[#106852] (issue: {es-issue}106387[#106387]) * Leverage ordinals in enrich lookup {es-pull}107449[#107449] * Serialize big array blocks {es-pull}106373[#106373] * Serialize big array vectors {es-pull}106327[#106327] * Specialize serialization for `ArrayVectors` {es-pull}105893[#105893] * Specialize serialization of array blocks {es-pull}106102[#106102] * Speed up serialization of `BytesRefArray` {es-pull}106053[#106053] * Support ST_CONTAINS and ST_WITHIN {es-pull}106503[#106503] * Support ST_INTERSECTS between geometry column and other geometry or string {es-pull}104907[#104907] (issue: {es-issue}104874[#104874]) Engine:: * Add metric for calculating index flush time excluding waiting on locks {es-pull}107196[#107196] Highlighting:: * Enable 'encoder' and 'tags_schema' highlighting settings at field level {es-pull}107224[#107224] (issue: {es-issue}94028[#94028]) ILM+SLM:: * Add a flag to re-enable writes on the final index after an ILM shrink action. {es-pull}107121[#107121] (issue: {es-issue}106599[#106599]) Indices APIs:: * Wait forever for `IndexTemplateRegistry` asset installation {es-pull}105985[#105985] Infra/CLI:: * Enhance search tier GC options {es-pull}106526[#106526] * Increase KDF iteration count in `KeyStoreWrapper` {es-pull}107107[#107107] Infra/Core:: * Add pluggable `BuildVersion` in `NodeMetadata` {es-pull}105757[#105757] Infra/Metrics:: * Infrastructure for metering the update requests {es-pull}105063[#105063] * `DocumentParsingObserver` to accept an `indexName` to allow skipping system indices {es-pull}107041[#107041] Infra/Scripting:: * String sha512() painless function {es-pull}99048[#99048] (issue: {es-issue}97691[#97691]) Ingest Node:: * Add support for the 'Anonymous IP' database to the geoip processor {es-pull}107287[#107287] (issue: {es-issue}90789[#90789]) * Add support for the 'Enterprise' database to the geoip processor {es-pull}107377[#107377] * Adding `cache_stats` to geoip stats API {es-pull}107334[#107334] * Support data streams in enrich policy indices {es-pull}107291[#107291] (issue: {es-issue}98836[#98836]) Machine Learning:: * Add GET `_inference` for all inference endpoints {es-pull}107517[#107517] * Added a timeout parameter to the inference API {es-pull}107242[#107242] * Enable retrying on 500 error response from Cohere text embedding API {es-pull}105797[#105797] Mapping:: * Make int8_hnsw our default index for new dense-vector fields {es-pull}106836[#106836] Ranking:: * Add retrievers using the parser-only approach {es-pull}105470[#105470] Search:: * Add Lucene spanish plural stemmer {es-pull}106952[#106952] * Add `modelId` and `modelText` to `KnnVectorQueryBuilder` {es-pull}106068[#106068] * Add a SIMD (Neon) optimised vector distance function for int8 {es-pull}106133[#106133] * Add transport version for search load autoscaling {es-pull}106377[#106377] * CCS with `minimize_roundtrips` performs incremental merges of each `SearchResponse` {es-pull}105781[#105781] * Track ongoing search tasks {es-pull}107129[#107129] Security:: * Invalidating cross cluster API keys requires `manage_security` {es-pull}107411[#107411] * Show owner `realm_type` for returned API keys {es-pull}105629[#105629] Snapshot/Restore:: * Add setting for max connections to S3 {es-pull}107533[#107533] * Distinguish different snapshot failures by log level {es-pull}105622[#105622] Stats:: * (API+) CAT Nodes alias for shard header to match CAT Allocation {es-pull}105847[#105847] * Add total size in bytes to doc stats {es-pull}106840[#106840] (issue: {es-issue}97670[#97670]) TSDB:: * Improve short-circuiting downsample execution {es-pull}106563[#106563] * Support non-keyword dimensions as routing fields in TSDB {es-pull}105501[#105501] * Text fields are stored by default in TSDB indices {es-pull}106338[#106338] (issue: {es-issue}97039[#97039]) Transform:: * Check node shutdown before fail {es-pull}107358[#107358] (issue: {es-issue}100891[#100891]) * Do not log error on node restart when the transform is already failed {es-pull}106171[#106171] (issue: {es-issue}106168[#106168]) [[feature-8.14.0]] [float] === New features Application:: * Allow `typed_keys` for search application Search API {es-pull}108007[#108007] * [Connector API] Support cleaning up sync jobs when deleting a connector {es-pull}107253[#107253] ES|QL:: * ESQL: Values aggregation function {es-pull}106065[#106065] (issue: {es-issue}103600[#103600]) * ESQL: allow sorting by expressions and not only regular fields {es-pull}107158[#107158] * Support ES|QL requests through the `NodeClient::execute` {es-pull}106244[#106244] Indices APIs:: * Add granular error list to alias action response {es-pull}106514[#106514] (issue: {es-issue}94478[#94478]) Machine Learning:: * Add Cohere rerank to `_inference` service {es-pull}106378[#106378] * Add support for Azure OpenAI embeddings to inference service {es-pull}107178[#107178] * Create default word based chunker {es-pull}107303[#107303] * Text structure endpoints to determine the structure of a list of messages and of an indexed field {es-pull}105660[#105660] Mapping:: * Flatten object mappings when subobjects is false {es-pull}103542[#103542] (issues: {es-issue}99860[#99860], {es-issue}103497[#103497]) Security:: * Get and Query API Key with profile uid {es-pull}106531[#106531] Vector Search:: * Adding support for hex-encoded byte vectors on knn-search {es-pull}105393[#105393] [[upgrade-8.14.0]] [float] === Upgrades Infra/Core:: * Upgrade jna to 5.12.1 {es-pull}105717[#105717] Ingest Node:: * Updating the tika version to 2.9.1 in the ingest attachment plugin {es-pull}106315[#106315] Network:: * Upgrade to Netty 4.1.107 {es-pull}105517[#105517] Packaging:: * Update bundled JDK to Java 22 (again) {es-pull}108654[#108654]