Add a cluster setting configuring `TDigestExecutionHint` (#96943)

* Initial import for TDigest forking.

* Fix MedianTest.

More work needed for TDigestPercentile*Tests and the TDigestTest (and
the rest of the tests) in the tdigest lib to pass.

* Fix Dist.

* Fix AVLTreeDigest.quantile to match Dist for uniform centroids.

* Update docs/changelog/96086.yaml

* Fix `MergingDigest.quantile` to match `Dist` on uniform distribution.

* Add merging to TDigestState.hashCode and .equals.

Remove wrong asserts from tests and MergingDigest.

* Fix style violations for tdigest library.

* Fix typo.

* Fix more style violations.

* Fix more style violations.

* Fix remaining style violations in tdigest library.

* Update results in docs based on the forked tdigest.

* Fix YAML tests in aggs module.

* Fix YAML tests in x-pack/plugin.

* Skip failing V7 compat tests in modules/aggregations.

* Fix TDigest library unittests.

Remove redundant serializing interfaces from the library.

* Remove YAML test versions for older releases.

These tests don't address compatibility issues in mixed cluster tests as
the latter contain a mix of older and newer nodes, so the output depends
on which node is picked as a data node since the forked TDigest library
is not backwards compatible (produces slightly different results).

* Fix test failures in docs and mixed cluster.

* Reduce buffer sizes in MergingDigest to avoid oom.

* Exclude more failing V7 compatibility tests.

* Update results for JdbcCsvSpecIT tests.

* Update results for JdbcDocCsvSpecIT tests.

* Revert unrelated change.

* More test fixes.

* Use version skips instead of blacklisting in mixed cluster tests.

* Switch TDigestState back to AVLTreeDigest.

* Update docs and tests with AVLTreeDigest output.

* Update flaky test.

* Remove dead code, esp around tracking of incoming data.

* Update docs/changelog/96086.yaml

* Delete docs/changelog/96086.yaml

* Remove explicit compression calls.

This was added to prevent concurrency tests from failing, but it leads
to reduces precision. Submit this to see if the concurrency tests are
still failing.

* Revert "Remove explicit compression calls."

This reverts commit 5352c96f656b9cdc63f0f673c983150bea35c7ed.

* Remove explicit compression calls to MedianAbsoluteDeviation input.

* Add unittests for AVL and merging digest accuracy.

* Fix spotless violations.

* Delete redundant tests and benchmarks.

* Fix spotless violation.

* Use the old implementation of AVLTreeDigest.

The latest library version is 50% slower and less accurate, as verified
by ComparisonTests.

* Update docs with latest percentile results.

* Update docs with latest percentile results.

* Remove repeated compression calls.

* Update more percentile results.

* Use approximate percentile values in integration tests.

This helps with mixed cluster tests, where some of the tests where
blocked.

* Fix expected percentile value in test.

* Revert in-place node updates in AVL tree.

Update quantile calculations between centroids and min/max values to
match v.3.2.

* Add SortingDigest and HybridDigest.

The SortingDigest tracks all samples in an ArrayList that
gets sorted for quantile calculations. This approach
provides perfectly accurate results and is the most
efficient implementation for up to millions of samples,
at the cost of bloated memory footprint.

The HybridDigest uses a SortingDigest for small sample
populations, then switches to a MergingDigest. This
approach combines to the best performance and results for
small sample counts with very good performance and
acceptable accuracy for effectively unbounded sample
counts.

* Remove deps to the 3.2 library.

* Remove unused licenses for tdigest.

* Revert changes for SortingDigest and HybridDigest.

These will be submitted in a follow-up PR for enabling MergingDigest.

* Remove unused Histogram classes and unit tests.

Delete dead and commented out code, make the remaining tests run
reasonably fast. Remove unused annotations, esp. SuppressWarnings.

* Remove Comparison class, not used.

* Revert "Revert changes for SortingDigest and HybridDigest."

This reverts commit 2336b11598a5b2e4ef284560db2469cb0a054849.

* Use HybridDigest as default tdigest implementation

Add SortingDigest as a simple structure for percentile calculations that
tracks all data points in a sorted array. This is a fast and perfectly
accurate solution that leads to bloated memory allocation.

Add HybridDigest that uses SortingDigest for small sample counts, then
switches to MergingDigest. This approach delivers extreme
performance and accuracy for small populations while scaling
indefinitely and maintaining acceptable performance and accuracy with
constant memory allocation (15kB by default).

Provide knobs to switch back to AVLTreeDigest, either per query or
through ClusterSettings.

* Small fixes.

* Add javadoc and tests.

* Add javadoc and tests.

* Remove special logic for singletons in the boundaries.

While this helps with the case where the digest contains only
singletons (perfect accuracy), it has a major issue problem
(non-monotonic quantile function) when the first singleton is followed
by a non-singleton centroid. It's preferable to revert to the old
version from 3.2; inaccuracies in a singleton-only digest should be
mitigated by using a sorted array for small sample counts.

* Revert changes to expected values in tests.

This is due to restoring quantile functions to match head.

* Revert changes to expected values in tests.

This is due to restoring quantile functions to match head.

* Tentatively restore percentile rank expected results.

* Use cdf version from 3.2

Update Dist.cdf to use interpolation, use the same cdf
version in AVLTreeDigest and MergingDigest.

* Revert "Tentatively restore percentile rank expected results."

This reverts commit 7718dbba594f068913dfb7b85f488ca25e5e47bf.

* Revert remaining changes compared to main.

* Revert excluded V7 compat tests.

* Exclude V7 compat tests still failing.

* Exclude V7 compat tests still failing.

* Remove ClusterSettings tentatively.

* Initial import for TDigest forking.

* Fix MedianTest.

More work needed for TDigestPercentile*Tests and the TDigestTest (and
the rest of the tests) in the tdigest lib to pass.

* Fix Dist.

* Fix AVLTreeDigest.quantile to match Dist for uniform centroids.

* Update docs/changelog/96086.yaml

* Fix `MergingDigest.quantile` to match `Dist` on uniform distribution.

* Add merging to TDigestState.hashCode and .equals.

Remove wrong asserts from tests and MergingDigest.

* Fix style violations for tdigest library.

* Fix typo.

* Fix more style violations.

* Fix more style violations.

* Fix remaining style violations in tdigest library.

* Update results in docs based on the forked tdigest.

* Fix YAML tests in aggs module.

* Fix YAML tests in x-pack/plugin.

* Skip failing V7 compat tests in modules/aggregations.

* Fix TDigest library unittests.

Remove redundant serializing interfaces from the library.

* Remove YAML test versions for older releases.

These tests don't address compatibility issues in mixed cluster tests as
the latter contain a mix of older and newer nodes, so the output depends
on which node is picked as a data node since the forked TDigest library
is not backwards compatible (produces slightly different results).

* Fix test failures in docs and mixed cluster.

* Reduce buffer sizes in MergingDigest to avoid oom.

* Exclude more failing V7 compatibility tests.

* Update results for JdbcCsvSpecIT tests.

* Update results for JdbcDocCsvSpecIT tests.

* Revert unrelated change.

* More test fixes.

* Use version skips instead of blacklisting in mixed cluster tests.

* Switch TDigestState back to AVLTreeDigest.

* Update docs and tests with AVLTreeDigest output.

* Update flaky test.

* Remove dead code, esp around tracking of incoming data.

* Remove explicit compression calls.

This was added to prevent concurrency tests from failing, but it leads
to reduces precision. Submit this to see if the concurrency tests are
still failing.

* Update docs/changelog/96086.yaml

* Delete docs/changelog/96086.yaml

* Revert "Remove explicit compression calls."

This reverts commit 5352c96f656b9cdc63f0f673c983150bea35c7ed.

* Remove explicit compression calls to MedianAbsoluteDeviation input.

* Add unittests for AVL and merging digest accuracy.

* Fix spotless violations.

* Delete redundant tests and benchmarks.

* Fix spotless violation.

* Use the old implementation of AVLTreeDigest.

The latest library version is 50% slower and less accurate, as verified
by ComparisonTests.

* Update docs with latest percentile results.

* Update docs with latest percentile results.

* Remove repeated compression calls.

* Update more percentile results.

* Use approximate percentile values in integration tests.

This helps with mixed cluster tests, where some of the tests where
blocked.

* Fix expected percentile value in test.

* Revert in-place node updates in AVL tree.

Update quantile calculations between centroids and min/max values to
match v.3.2.

* Add SortingDigest and HybridDigest.

The SortingDigest tracks all samples in an ArrayList that
gets sorted for quantile calculations. This approach
provides perfectly accurate results and is the most
efficient implementation for up to millions of samples,
at the cost of bloated memory footprint.

The HybridDigest uses a SortingDigest for small sample
populations, then switches to a MergingDigest. This
approach combines to the best performance and results for
small sample counts with very good performance and
acceptable accuracy for effectively unbounded sample
counts.

* Remove deps to the 3.2 library.

* Remove unused licenses for tdigest.

* Revert changes for SortingDigest and HybridDigest.

These will be submitted in a follow-up PR for enabling MergingDigest.

* Remove unused Histogram classes and unit tests.

Delete dead and commented out code, make the remaining tests run
reasonably fast. Remove unused annotations, esp. SuppressWarnings.

* Remove Comparison class, not used.

* Revert "Revert changes for SortingDigest and HybridDigest."

This reverts commit 2336b11598a5b2e4ef284560db2469cb0a054849.

* Use HybridDigest as default tdigest implementation

Add SortingDigest as a simple structure for percentile calculations that
tracks all data points in a sorted array. This is a fast and perfectly
accurate solution that leads to bloated memory allocation.

Add HybridDigest that uses SortingDigest for small sample counts, then
switches to MergingDigest. This approach delivers extreme
performance and accuracy for small populations while scaling
indefinitely and maintaining acceptable performance and accuracy with
constant memory allocation (15kB by default).

Provide knobs to switch back to AVLTreeDigest, either per query or
through ClusterSettings.

* Add javadoc and tests.

* Remove ClusterSettings tentatively.

* Restore bySize function in TDigest and subclasses.

* Update Dist.cdf to match the rest.

Update tests.

* Revert outdated test changes.

* Revert outdated changes.

* Small fixes.

* Update docs/changelog/96794.yaml

* Make HybridDigest the default implementation.

* Update boxplot documentation.

* Restore AVLTreeDigest as the default in TDigestState.

TDigest.createHybridDigest nw returns the right type.
The switch in TDigestState will happen in a separate PR
as it requires many test updates.

* Use execution_hint in tdigest spec.

* Fix Dist.cdf for empty digest.

* Pass ClusterSettings through SearchExecutionContext.

* Bump up TransportVersion.

* Bump up TransportVersion for real.

* HybridDigest uses its final implementation during deserialization.

* Restore the right TransportVersion in TDigestState.read

* Add dummy SearchExecutionContext factory for tests.

* Use TDigestExecutionHint instead of strings.

* Remove check for null context.

* Add link to TDigest javadoc.

* Use NodeSettings directly.

* Init executionHint to null, set before using.

* Update docs/changelog/96943.yaml

* Pass initialized executionHint to createEmptyPercentileRanksAggregator.

* Initialize TDigestExecutionHint.SETTING to "DEFAULT".

* Initialize TDigestExecutionHint to null.

* Use readOptionalWriteable/writeOptionalWriteable.

Move test-only SearchExecutionContext method in helper class under
test.

* Bump up TransportVersion.

* Small fixes.

benchmarks/src/main/java/org/elasticsearch/benchmark/search/QueryParserHelperBenchmark.java

@@ -24,6 +24,7 @@ import org.elasticsearch.common.bytes.BytesArray;
 import org.elasticsearch.common.compress.CompressedXContent;
 import org.elasticsearch.common.io.stream.NamedWriteableRegistry;
 import org.elasticsearch.common.lucene.Lucene;
+import org.elasticsearch.common.settings.ClusterSettings;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.common.xcontent.LoggingDeprecationHandler;
 import org.elasticsearch.core.IOUtils;
@@ -138,6 +139,7 @@ public class QueryParserHelperBenchmark {
             0,
             0,
             mapperService.getIndexSettings(),
+            ClusterSettings.createBuiltInClusterSettings(),
             null,
             (ft, fdc) -> ft.fielddataBuilder(fdc).build(new IndexFieldDataCache.None(), new NoneCircuitBreakerService()),
             mapperService,

benchmarks/src/main/java/org/elasticsearch/benchmark/search/aggregations/AggConstructionContentionBenchmark.java

@@ -269,6 +269,11 @@ public class AggConstructionContentionBenchmark {
             throw new UnsupportedOperationException();
         }
 
+        @Override
+        public ClusterSettings getClusterSettings() {
+            throw new UnsupportedOperationException();
+        }
+
         @Override
         public Optional<SortAndFormats> buildSort(List<SortBuilder<?>> sortBuilders) throws IOException {
             throw new UnsupportedOperationException();

docs/changelog/96943.yaml

@@ -0,0 +1,5 @@
+pr: 96943
+summary: Add cluster setting to `SearchExecutionContext` to configure `TDigestExecutionHint`
+area: Aggregations
+type: enhancement
+issues: []

server/src/main/java/org/elasticsearch/TransportVersion.java

@@ -142,9 +142,10 @@ public record TransportVersion(int id) implements Comparable<TransportVersion> {
     public static final TransportVersion V_8_500_015 = registerTransportVersion(8_500_015, "651216c9-d54f-4189-9fe1-48d82d276863");
     public static final TransportVersion V_8_500_016 = registerTransportVersion(8_500_016, "492C94FB-AAEA-4C9E-8375-BDB67A398584");
     public static final TransportVersion V_8_500_017 = registerTransportVersion(8_500_017, "0EDCB5BA-049C-443C-8AB1-5FA58FB996FB");
+    public static final TransportVersion V_8_500_018 = registerTransportVersion(8_500_018, "827C32CE-33D9-4AC3-A773-8FB768F59EAF");
 
     private static class CurrentHolder {
-        private static final TransportVersion CURRENT = findCurrent(V_8_500_017);
+        private static final TransportVersion CURRENT = findCurrent(V_8_500_018);
 
         // finds the pluggable current version, or uses the given fallback
         private static TransportVersion findCurrent(TransportVersion fallback) {

server/src/main/java/org/elasticsearch/common/settings/ClusterSettings.java

@@ -110,6 +110,7 @@ import org.elasticsearch.script.ScriptService;
 import org.elasticsearch.search.SearchModule;
 import org.elasticsearch.search.SearchService;
 import org.elasticsearch.search.aggregations.MultiBucketConsumerService;
+import org.elasticsearch.search.aggregations.metrics.TDigestExecutionHint;
 import org.elasticsearch.search.fetch.subphase.highlight.FastVectorHighlighter;
 import org.elasticsearch.snapshots.InternalSnapshotsInfoService;
 import org.elasticsearch.snapshots.RestoreService;
@@ -576,7 +577,8 @@ public final class ClusterSettings extends AbstractScopedSettings {
         IndicesClusterStateService.SHARD_LOCK_RETRY_INTERVAL_SETTING,
         IndicesClusterStateService.SHARD_LOCK_RETRY_TIMEOUT_SETTING,
         IngestSettings.GROK_WATCHDOG_INTERVAL,
-        IngestSettings.GROK_WATCHDOG_MAX_EXECUTION_TIME
+        IngestSettings.GROK_WATCHDOG_MAX_EXECUTION_TIME,
+        TDigestExecutionHint.SETTING
     ).filter(Objects::nonNull).collect(Collectors.toSet());
 
     static List<SettingUpgrader<?>> BUILT_IN_SETTING_UPGRADERS = Collections.emptyList();

server/src/main/java/org/elasticsearch/index/IndexService.java

@@ -650,6 +650,7 @@ public class IndexService extends AbstractIndexComponent implements IndicesClust
             shardId,
             shardRequestIndex,
             indexSettings,
+            clusterService.getClusterSettings(),
             indexCache.bitsetFilterCache(),
             indexFieldData::getForField,
             mapperService(),

server/src/main/java/org/elasticsearch/index/query/SearchExecutionContext.java

@@ -25,6 +25,7 @@ import org.elasticsearch.common.ParsingException;
 import org.elasticsearch.common.io.stream.NamedWriteableRegistry;
 import org.elasticsearch.common.lucene.search.Queries;
 import org.elasticsearch.common.regex.Regex;
+import org.elasticsearch.common.settings.ClusterSettings;
 import org.elasticsearch.core.CheckedFunction;
 import org.elasticsearch.index.Index;
 import org.elasticsearch.index.IndexSettings;
@@ -89,7 +90,8 @@ public class SearchExecutionContext extends QueryRewriteContext {
     private final SimilarityService similarityService;
     private final BitsetFilterCache bitsetFilterCache;
     private final BiFunction<MappedFieldType, FieldDataContext, IndexFieldData<?>> indexFieldDataLookup;
-    private SearchLookup lookup = null;
+    private SearchLookup lookup;
+    private ClusterSettings clusterSettings;
 
     private final int shardId;
     private final int shardRequestIndex;
@@ -108,6 +110,7 @@ public class SearchExecutionContext extends QueryRewriteContext {
         int shardId,
         int shardRequestIndex,
         IndexSettings indexSettings,
+        ClusterSettings clusterSettings,
         BitsetFilterCache bitsetFilterCache,
         BiFunction<MappedFieldType, FieldDataContext, IndexFieldData<?>> indexFieldDataLookup,
         MapperService mapperService,
@@ -129,6 +132,7 @@ public class SearchExecutionContext extends QueryRewriteContext {
             shardId,
             shardRequestIndex,
             indexSettings,
+            clusterSettings,
             bitsetFilterCache,
             indexFieldDataLookup,
             mapperService,
@@ -157,6 +161,7 @@ public class SearchExecutionContext extends QueryRewriteContext {
             source.shardId,
             source.shardRequestIndex,
             source.indexSettings,
+            source.clusterSettings,
             source.bitsetFilterCache,
             source.indexFieldDataLookup,
             source.mapperService,
@@ -181,6 +186,7 @@ public class SearchExecutionContext extends QueryRewriteContext {
         int shardId,
         int shardRequestIndex,
         IndexSettings indexSettings,
+        ClusterSettings clusterSettings,
         BitsetFilterCache bitsetFilterCache,
         BiFunction<MappedFieldType, FieldDataContext, IndexFieldData<?>> indexFieldDataLookup,
         MapperService mapperService,
@@ -222,6 +228,7 @@ public class SearchExecutionContext extends QueryRewriteContext {
         this.indexFieldDataLookup = indexFieldDataLookup;
         this.nestedScope = new NestedScope();
         this.searcher = searcher;
+        this.clusterSettings = clusterSettings;
     }
 
     private void reset() {
@@ -598,6 +605,14 @@ public class SearchExecutionContext extends QueryRewriteContext {
         return this;
     }
 
+    /**
+     * Returns the cluster settings for this context. This might return null if the
+     * context has not cluster scope.
+     */
+    public ClusterSettings getClusterSettings() {
+        return clusterSettings;
+    }
+
     /** Return the current {@link IndexReader}, or {@code null} if no index reader is available,
      *  for instance if this rewrite context is used to index queries (percolation).
      */

server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregationBuilder.java

@@ -54,7 +54,7 @@ public class MedianAbsoluteDeviationAggregationBuilder extends SingleMetricAggre
     }
 
     private double compression = 1000d;
-    private TDigestExecutionHint executionHint = TDigestExecutionHint.DEFAULT;
+    private TDigestExecutionHint executionHint = null;
 
     public MedianAbsoluteDeviationAggregationBuilder(String name) {
         super(name);
@@ -63,9 +63,11 @@ public class MedianAbsoluteDeviationAggregationBuilder extends SingleMetricAggre
     public MedianAbsoluteDeviationAggregationBuilder(StreamInput in) throws IOException {
         super(in);
         compression = in.readDouble();
-        executionHint = in.getTransportVersion().onOrAfter(TransportVersion.V_8_500_014)
-            ? TDigestExecutionHint.readFrom(in)
-            : TDigestExecutionHint.HIGH_ACCURACY;
+        if (in.getTransportVersion().onOrAfter(TransportVersion.V_8_500_018)) {
+            executionHint = in.readOptionalWriteable(TDigestExecutionHint::readFrom);
+        } else {
+            executionHint = TDigestExecutionHint.HIGH_ACCURACY;
+        }
     }
 
     protected MedianAbsoluteDeviationAggregationBuilder(
@@ -124,8 +126,8 @@ public class MedianAbsoluteDeviationAggregationBuilder extends SingleMetricAggre
     @Override
     protected void innerWriteTo(StreamOutput out) throws IOException {
         out.writeDouble(compression);
-        if (out.getTransportVersion().onOrAfter(TransportVersion.V_8_500_014)) {
-            executionHint.writeTo(out);
+        if (out.getTransportVersion().onOrAfter(TransportVersion.V_8_500_018)) {
+            out.writeOptionalWriteable(executionHint);
         }
     }
 
@@ -140,6 +142,9 @@ public class MedianAbsoluteDeviationAggregationBuilder extends SingleMetricAggre
         MedianAbsoluteDeviationAggregatorSupplier aggregatorSupplier = context.getValuesSourceRegistry()
             .getAggregator(REGISTRY_KEY, config);
 
+        if (executionHint == null) {
+            executionHint = TDigestExecutionHint.parse(context.getClusterSettings().get(TDigestExecutionHint.SETTING));
+        }
         return new MedianAbsoluteDeviationAggregatorFactory(
             name,
             config,
@@ -156,7 +161,9 @@ public class MedianAbsoluteDeviationAggregationBuilder extends SingleMetricAggre
     @Override
     protected XContentBuilder doXContentBody(XContentBuilder builder, Params params) throws IOException {
         builder.field(COMPRESSION_FIELD.getPreferredName(), compression);
-        builder.field(EXECUTION_HINT_FIELD.getPreferredName(), executionHint);
+        if (executionHint != null) {
+            builder.field(EXECUTION_HINT_FIELD.getPreferredName(), executionHint);
+        }
         return builder;
     }
 
@@ -171,7 +178,7 @@ public class MedianAbsoluteDeviationAggregationBuilder extends SingleMetricAggre
         if (obj == null || getClass() != obj.getClass()) return false;
         if (super.equals(obj) == false) return false;
         MedianAbsoluteDeviationAggregationBuilder other = (MedianAbsoluteDeviationAggregationBuilder) obj;
-        return Objects.equals(compression, other.compression) && executionHint.equals(other.executionHint);
+        return Objects.equals(compression, other.compression) && Objects.equals(executionHint, other.executionHint);
     }
 
     @Override

server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesConfig.java

@@ -124,14 +124,14 @@ public abstract class PercentilesConfig implements ToXContent, Writeable {
         static final double DEFAULT_COMPRESSION = 100.0;
         private double compression;
 
-        private TDigestExecutionHint executionHint = TDigestExecutionHint.DEFAULT;
+        private TDigestExecutionHint executionHint;
 
         public TDigest() {
             this(DEFAULT_COMPRESSION);
         }
 
         public TDigest(double compression) {
-            this(compression, TDigestExecutionHint.DEFAULT);
+            this(compression, null);
         }
 
         public TDigest(double compression, TDigestExecutionHint executionHint) {
@@ -143,8 +143,8 @@ public abstract class PercentilesConfig implements ToXContent, Writeable {
         TDigest(StreamInput in) throws IOException {
             this(
                 in.readDouble(),
-                in.getTransportVersion().onOrAfter(TransportVersion.V_8_500_014)
-                    ? TDigestExecutionHint.readFrom(in)
+                in.getTransportVersion().onOrAfter(TransportVersion.V_8_500_018)
+                    ? in.readOptionalWriteable(TDigestExecutionHint::readFrom)
                     : TDigestExecutionHint.HIGH_ACCURACY
             );
         }
@@ -164,7 +164,10 @@ public abstract class PercentilesConfig implements ToXContent, Writeable {
             this.executionHint = TDigestExecutionHint.parse(executionHint);
         }
 
-        public TDigestExecutionHint getExecutionHint() {
+        public TDigestExecutionHint getExecutionHint(AggregationContext context) {
+            if (executionHint == null) {
+                executionHint = TDigestExecutionHint.parse(context.getClusterSettings().get(TDigestExecutionHint.SETTING));
+            }
             return executionHint;
         }
 
@@ -186,7 +189,7 @@ public abstract class PercentilesConfig implements ToXContent, Writeable {
                 parent,
                 values,
                 compression,
-                executionHint,
+                getExecutionHint(context),
                 keyed,
                 formatter,
                 metadata
@@ -222,7 +225,7 @@ public abstract class PercentilesConfig implements ToXContent, Writeable {
                 parent,
                 values,
                 compression,
-                executionHint,
+                getExecutionHint(context),
                 keyed,
                 formatter,
                 metadata
@@ -237,15 +240,16 @@ public abstract class PercentilesConfig implements ToXContent, Writeable {
             DocValueFormat formatter,
             Map<String, Object> metadata
         ) {
-            return InternalTDigestPercentileRanks.empty(name, values, compression, executionHint, keyed, formatter, metadata);
+            TDigestExecutionHint hint = executionHint == null ? TDigestExecutionHint.DEFAULT : executionHint;
+            return InternalTDigestPercentileRanks.empty(name, values, compression, hint, keyed, formatter, metadata);
         }
 
         @Override
         public void writeTo(StreamOutput out) throws IOException {
             super.writeTo(out);
             out.writeDouble(compression);
-            if (out.getTransportVersion().onOrAfter(TransportVersion.V_8_500_014)) {
-                executionHint.writeTo(out);
+            if (out.getTransportVersion().onOrAfter(TransportVersion.V_8_500_018)) {
+                out.writeOptionalWriteable(executionHint);
             }
         }
 
@@ -253,6 +257,9 @@ public abstract class PercentilesConfig implements ToXContent, Writeable {
         public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
             builder.startObject(getMethod().toString());
             builder.field(PercentilesMethod.COMPRESSION_FIELD.getPreferredName(), compression);
+            if (executionHint != null) {
+                builder.field(PercentilesMethod.EXECUTION_HINT_FIELD.getPreferredName(), executionHint);
+            }
             builder.endObject();
             return builder;
         }
@@ -264,7 +271,7 @@ public abstract class PercentilesConfig implements ToXContent, Writeable {
             if (super.equals(obj) == false) return false;
 
             TDigest other = (TDigest) obj;
-            return compression == other.getCompression() && executionHint.equals(other.getExecutionHint());
+            return compression == other.getCompression() && Objects.equals(executionHint, other.executionHint);
         }
 
         @Override

server/src/main/java/org/elasticsearch/search/aggregations/metrics/TDigestExecutionHint.java

@@ -11,6 +11,7 @@ package org.elasticsearch.search.aggregations.metrics;
 import org.elasticsearch.common.io.stream.StreamInput;
 import org.elasticsearch.common.io.stream.StreamOutput;
 import org.elasticsearch.common.io.stream.Writeable;
+import org.elasticsearch.common.settings.Setting;
 
 import java.io.IOException;
 
@@ -21,6 +22,14 @@ public enum TDigestExecutionHint implements Writeable {
     DEFAULT(0),        // Use a TDigest that is optimized for performance, with a small penalty in accuracy.
     HIGH_ACCURACY(1);  // Use a TDigest that is optimize for accuracy, at the expense of performance.
 
+    public static final Setting<String> SETTING = Setting.simpleString(
+        "search.aggs.tdigest_execution_hint",
+        TDigestExecutionHint.DEFAULT.toString(),
+        TDigestExecutionHint::parse,
+        Setting.Property.NodeScope,
+        Setting.Property.Dynamic
+    );
+
     TDigestExecutionHint(int id) {
         this.id = id;
     }

server/src/main/java/org/elasticsearch/search/aggregations/support/AggregationContext.java

@@ -15,6 +15,7 @@ import org.apache.lucene.search.IndexSearcher;
 import org.apache.lucene.search.Query;
 import org.elasticsearch.common.breaker.CircuitBreaker;
 import org.elasticsearch.common.breaker.PreallocatedCircuitBreakerService;
+import org.elasticsearch.common.settings.ClusterSettings;
 import org.elasticsearch.common.util.BigArrays;
 import org.elasticsearch.core.Nullable;
 import org.elasticsearch.core.Releasable;
@@ -203,6 +204,11 @@ public abstract class AggregationContext implements Releasable {
      */
     public abstract IndexSettings getIndexSettings();
 
+    /**
+     * The settings for the cluster against which this search is running.
+     */
+    public abstract ClusterSettings getClusterSettings();
+
     /**
      * Compile a sort.
      */
@@ -500,6 +506,11 @@ public abstract class AggregationContext implements Releasable {
             return context.getIndexSettings();
         }
 
+        @Override
+        public ClusterSettings getClusterSettings() {
+            return context.getClusterSettings();
+        }
+
         @Override
         public Optional<SortAndFormats> buildSort(List<SortBuilder<?>> sortBuilders) throws IOException {
             return SortBuilder.buildSort(sortBuilders, context);

server/src/test/java/org/elasticsearch/cluster/metadata/MetadataCreateIndexServiceTests.java

@@ -45,8 +45,8 @@ import org.elasticsearch.index.IndexSettingProviders;
 import org.elasticsearch.index.IndexSettings;
 import org.elasticsearch.index.IndexVersion;
 import org.elasticsearch.index.mapper.MapperService;
-import org.elasticsearch.index.mapper.MappingLookup;
 import org.elasticsearch.index.query.SearchExecutionContext;
+import org.elasticsearch.index.query.SearchExecutionContextHelper;
 import org.elasticsearch.indices.EmptySystemIndices;
 import org.elasticsearch.indices.InvalidAliasNameException;
 import org.elasticsearch.indices.InvalidIndexNameException;
@@ -82,7 +82,6 @@ import java.util.function.Consumer;
 import java.util.stream.Collectors;
 import java.util.stream.Stream;
 
-import static java.util.Collections.emptyMap;
 import static org.elasticsearch.cluster.metadata.IndexMetadata.INDEX_NUMBER_OF_ROUTING_SHARDS_SETTING;
 import static org.elasticsearch.cluster.metadata.IndexMetadata.INDEX_NUMBER_OF_SHARDS_SETTING;
 import static org.elasticsearch.cluster.metadata.IndexMetadata.INDEX_READ_ONLY_BLOCK;
@@ -116,26 +115,10 @@ public class MetadataCreateIndexServiceTests extends ESTestCase {
     public void setupCreateIndexRequestAndAliasValidator() {
         request = new CreateIndexClusterStateUpdateRequest("create index", "test", "test");
         Settings indexSettings = indexSettings(Version.CURRENT, 1, 1).build();
-        searchExecutionContext = new SearchExecutionContext(
-            0,
-            0,
+        searchExecutionContext = SearchExecutionContextHelper.createSimple(
             new IndexSettings(IndexMetadata.builder("test").settings(indexSettings).build(), indexSettings),
-            null,
-            null,
-            null,
-            MappingLookup.EMPTY,
-            null,
-            null,
             parserConfig(),
-            writableRegistry(),
-            null,
-            null,
-            () -> randomNonNegativeLong(),
-            null,
-            null,
-            () -> true,
-            null,
-            emptyMap()
+            writableRegistry()
         );
     }
 

server/src/test/java/org/elasticsearch/index/mapper/DateFieldTypeTests.java

@@ -24,6 +24,7 @@ import org.apache.lucene.search.Query;
 import org.apache.lucene.store.Directory;
 import org.elasticsearch.Version;
 import org.elasticsearch.cluster.metadata.IndexMetadata;
+import org.elasticsearch.common.settings.ClusterSettings;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.common.time.DateFormatter;
 import org.elasticsearch.common.time.DateFormatters;
@@ -39,6 +40,7 @@ import org.elasticsearch.index.mapper.MappedFieldType.Relation;
 import org.elasticsearch.index.query.DateRangeIncludingNowQuery;
 import org.elasticsearch.index.query.QueryRewriteContext;
 import org.elasticsearch.index.query.SearchExecutionContext;
+import org.elasticsearch.index.query.SearchExecutionContextHelper;
 import org.elasticsearch.script.field.DateNanosDocValuesField;
 import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
 
@@ -49,8 +51,6 @@ import java.time.ZoneOffset;
 import java.util.Collections;
 import java.util.List;
 
-import static java.util.Collections.emptyMap;
-
 public class DateFieldTypeTests extends FieldTypeTestCase {
 
     private static final long nowInMillis = 0;
@@ -171,26 +171,10 @@ public class DateFieldTypeTests extends FieldTypeTestCase {
 
     public void testTermQuery() {
         Settings indexSettings = indexSettings(Version.CURRENT, 1, 1).build();
-        SearchExecutionContext context = new SearchExecutionContext(
-            0,
-            0,
+        SearchExecutionContext context = SearchExecutionContextHelper.createSimple(
             new IndexSettings(IndexMetadata.builder("foo").settings(indexSettings).build(), indexSettings),
-            null,
-            null,
-            null,
-            MappingLookup.EMPTY,
-            null,
-            null,
             parserConfig(),
-            writableRegistry(),
-            null,
-            null,
-            () -> nowInMillis,
-            null,
-            null,
-            () -> true,
-            null,
-            emptyMap()
+            writableRegistry()
         );
         MappedFieldType ft = new DateFieldType("field");
         String date = "2015-10-12T14:10:55";
@@ -226,6 +210,7 @@ public class DateFieldTypeTests extends FieldTypeTestCase {
             0,
             0,
             new IndexSettings(IndexMetadata.builder("foo").settings(indexSettings).build(), indexSettings),
+            ClusterSettings.createBuiltInClusterSettings(),
             null,
             null,
             null,
@@ -241,7 +226,7 @@ public class DateFieldTypeTests extends FieldTypeTestCase {
             null,
             () -> true,
             null,
-            emptyMap()
+            Collections.emptyMap()
         );
         MappedFieldType ft = new DateFieldType("field");
         String date1 = "2015-10-12T14:10:55";
@@ -301,27 +286,7 @@ public class DateFieldTypeTests extends FieldTypeTestCase {
         IndexMetadata indexMetadata = new IndexMetadata.Builder("index").settings(settings).build();
         IndexSettings indexSettings = new IndexSettings(indexMetadata, settings);
 
-        SearchExecutionContext context = new SearchExecutionContext(
-            0,
-            0,
-            indexSettings,
-            null,
-            null,
-            null,
-            MappingLookup.EMPTY,
-            null,
-            null,
-            parserConfig(),
-            writableRegistry(),
-            null,
-            null,
-            () -> 0L,
-            null,
-            null,
-            () -> true,
-            null,
-            emptyMap()
-        );
+        SearchExecutionContext context = SearchExecutionContextHelper.createSimple(indexSettings, parserConfig(), writableRegistry());
 
         MappedFieldType ft = new DateFieldType("field");
         String date1 = "2015-10-12T14:10:55";

server/src/test/java/org/elasticsearch/index/mapper/FieldNamesFieldTypeTests.java

@@ -15,13 +15,13 @@ import org.elasticsearch.cluster.metadata.IndexMetadata;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.index.IndexSettings;
 import org.elasticsearch.index.query.SearchExecutionContext;
+import org.elasticsearch.index.query.SearchExecutionContextHelper;
 import org.elasticsearch.test.ESTestCase;
 
 import java.util.List;
 import java.util.stream.Stream;
 
 import static java.util.Collections.emptyList;
-import static java.util.Collections.emptyMap;
 
 public class FieldNamesFieldTypeTests extends ESTestCase {
 
@@ -36,27 +36,7 @@ public class FieldNamesFieldTypeTests extends ESTestCase {
         );
         List<FieldMapper> mappers = Stream.of(fieldNamesFieldType, fieldType).<FieldMapper>map(MockFieldMapper::new).toList();
         MappingLookup mappingLookup = MappingLookup.fromMappers(Mapping.EMPTY, mappers, emptyList(), emptyList());
-        SearchExecutionContext searchExecutionContext = new SearchExecutionContext(
-            0,
-            0,
-            indexSettings,
-            null,
-            null,
-            null,
-            mappingLookup,
-            null,
-            null,
-            null,
-            null,
-            null,
-            null,
-            () -> 0L,
-            null,
-            null,
-            () -> true,
-            null,
-            emptyMap()
-        );
+        SearchExecutionContext searchExecutionContext = SearchExecutionContextHelper.createSimple(indexSettings, null, null);
         Query termQuery = fieldNamesFieldType.termQuery("field_name", searchExecutionContext);
         assertEquals(new TermQuery(new Term(FieldNamesFieldMapper.CONTENT_TYPE, "field_name")), termQuery);
         assertWarnings("terms query on the _field_names field is deprecated and will be removed, use exists query instead");

server/src/test/java/org/elasticsearch/index/mapper/IndexFieldTypeTests.java

@@ -12,15 +12,16 @@ import org.apache.lucene.search.MatchNoDocsQuery;
 import org.elasticsearch.Version;
 import org.elasticsearch.cluster.metadata.IndexMetadata;
 import org.elasticsearch.common.regex.Regex;
+import org.elasticsearch.common.settings.ClusterSettings;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.index.IndexSettings;
 import org.elasticsearch.index.query.QueryShardException;
 import org.elasticsearch.index.query.SearchExecutionContext;
 import org.elasticsearch.test.ESTestCase;
 
+import java.util.Collections;
 import java.util.function.Predicate;
 
-import static java.util.Collections.emptyMap;
 import static org.hamcrest.Matchers.containsString;
 
 public class IndexFieldTypeTests extends ESTestCase {
@@ -64,6 +65,7 @@ public class IndexFieldTypeTests extends ESTestCase {
             0,
             0,
             indexSettings,
+            ClusterSettings.createBuiltInClusterSettings(),
             null,
             null,
             null,
@@ -79,7 +81,7 @@ public class IndexFieldTypeTests extends ESTestCase {
             indexNameMatcher,
             () -> true,
             null,
-            emptyMap()
+            Collections.emptyMap()
         );
     }
 }

server/src/test/java/org/elasticsearch/index/mapper/NumberFieldTypeTests.java

@@ -44,6 +44,7 @@ import org.elasticsearch.index.mapper.MappedFieldType.Relation;
 import org.elasticsearch.index.mapper.NumberFieldMapper.NumberFieldType;
 import org.elasticsearch.index.mapper.NumberFieldMapper.NumberType;
 import org.elasticsearch.index.query.SearchExecutionContext;
+import org.elasticsearch.index.query.SearchExecutionContextHelper;
 import org.elasticsearch.script.ScriptCompiler;
 import org.elasticsearch.search.MultiValueMode;
 import org.elasticsearch.xcontent.XContentBuilder;
@@ -60,7 +61,6 @@ import java.util.Collections;
 import java.util.List;
 import java.util.function.Supplier;
 
-import static java.util.Collections.emptyMap;
 import static org.hamcrest.Matchers.containsString;
 import static org.hamcrest.Matchers.equalTo;
 import static org.hamcrest.Matchers.instanceOf;
@@ -611,27 +611,7 @@ public class NumberFieldTypeTests extends FieldTypeTestCase {
         DirectoryReader reader = DirectoryReader.open(w);
         IndexSearcher searcher = newSearcher(reader);
 
-        SearchExecutionContext context = new SearchExecutionContext(
-            0,
-            0,
-            indexSettings,
-            null,
-            null,
-            null,
-            MappingLookup.EMPTY,
-            null,
-            null,
-            parserConfig(),
-            writableRegistry(),
-            null,
-            null,
-            () -> 0L,
-            null,
-            null,
-            () -> true,
-            null,
-            emptyMap()
-        );
+        SearchExecutionContext context = SearchExecutionContextHelper.createSimple(indexSettings, parserConfig(), writableRegistry());
 
         final int iters = 10;
         for (int iter = 0; iter < iters; ++iter) {

server/src/test/java/org/elasticsearch/index/mapper/RangeFieldTypeTests.java

@@ -30,6 +30,7 @@ import org.elasticsearch.index.mapper.DateFieldMapper.DateFieldType;
 import org.elasticsearch.index.mapper.RangeFieldMapper.RangeFieldType;
 import org.elasticsearch.index.query.QueryShardException;
 import org.elasticsearch.index.query.SearchExecutionContext;
+import org.elasticsearch.index.query.SearchExecutionContextHelper;
 import org.elasticsearch.lucene.queries.BinaryDocValuesRangeQuery;
 import org.elasticsearch.test.IndexSettingsModule;
 import org.junit.Before;
@@ -43,7 +44,6 @@ import java.util.Collections;
 import java.util.List;
 import java.util.Map;
 
-import static java.util.Collections.emptyMap;
 import static org.hamcrest.Matchers.containsString;
 import static org.hamcrest.Matchers.instanceOf;
 
@@ -203,27 +203,7 @@ public class RangeFieldTypeTests extends FieldTypeTestCase {
     private SearchExecutionContext createContext() {
         Settings indexSettings = Settings.builder().put(IndexMetadata.SETTING_VERSION_CREATED, Version.CURRENT).build();
         IndexSettings idxSettings = IndexSettingsModule.newIndexSettings(randomAlphaOfLengthBetween(1, 10), indexSettings);
-        return new SearchExecutionContext(
-            0,
-            0,
-            idxSettings,
-            null,
-            null,
-            null,
-            MappingLookup.EMPTY,
-            null,
-            null,
-            parserConfig(),
-            writableRegistry(),
-            null,
-            null,
-            () -> nowInMillis,
-            null,
-            null,
-            () -> true,
-            null,
-            emptyMap()
-        );
+        return SearchExecutionContextHelper.createSimple(idxSettings, parserConfig(), writableRegistry());
     }
 
     public void testDateRangeQueryUsingMappingFormat() {

server/src/test/java/org/elasticsearch/index/query/RangeQueryRewriteTests.java

@@ -13,6 +13,7 @@ import org.apache.lucene.index.MultiReader;
 import org.apache.lucene.search.IndexSearcher;
 import org.elasticsearch.common.Strings;
 import org.elasticsearch.common.compress.CompressedXContent;
+import org.elasticsearch.common.settings.ClusterSettings;
 import org.elasticsearch.index.IndexService;
 import org.elasticsearch.index.mapper.MappedFieldType.Relation;
 import org.elasticsearch.index.mapper.MapperService.MergeReason;
@@ -33,6 +34,7 @@ public class RangeQueryRewriteTests extends ESSingleNodeTestCase {
             0,
             0,
             indexService.getIndexSettings(),
+            ClusterSettings.createBuiltInClusterSettings(),
             null,
             null,
             indexService.mapperService(),
@@ -73,6 +75,7 @@ public class RangeQueryRewriteTests extends ESSingleNodeTestCase {
             0,
             0,
             indexService.getIndexSettings(),
+            ClusterSettings.createBuiltInClusterSettings(),
             null,
             null,
             indexService.mapperService(),
@@ -115,6 +118,7 @@ public class RangeQueryRewriteTests extends ESSingleNodeTestCase {
             0,
             0,
             indexService.getIndexSettings(),
+            ClusterSettings.createBuiltInClusterSettings(),
             null,
             null,
             indexService.mapperService(),

server/src/test/java/org/elasticsearch/index/query/SearchExecutionContextTests.java

@@ -28,6 +28,7 @@ import org.elasticsearch.Version;
 import org.elasticsearch.cluster.metadata.IndexMetadata;
 import org.elasticsearch.common.Explicit;
 import org.elasticsearch.common.io.stream.NamedWriteableRegistry;
+import org.elasticsearch.common.settings.ClusterSettings;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.common.util.BigArrays;
 import org.elasticsearch.index.IndexSettings;
@@ -95,7 +96,6 @@ import java.util.function.Function;
 import java.util.function.Supplier;
 import java.util.stream.Collectors;
 
-import static java.util.Collections.emptyMap;
 import static java.util.Collections.singletonMap;
 import static org.hamcrest.Matchers.equalTo;
 import static org.hamcrest.Matchers.instanceOf;
@@ -176,26 +176,10 @@ public class SearchExecutionContextTests extends ESTestCase {
         IndexMetadata indexMetadata = new IndexMetadata.Builder("index").settings(settings).build();
 
         IndexSettings indexSettings = new IndexSettings(indexMetadata, settings);
-        SearchExecutionContext context = new SearchExecutionContext(
-            0,
-            0,
+        SearchExecutionContext context = SearchExecutionContextHelper.createSimple(
             indexSettings,
-            null,
-            null,
-            null,
-            MappingLookup.EMPTY,
-            null,
-            null,
             XContentParserConfiguration.EMPTY,
-            new NamedWriteableRegistry(Collections.emptyList()),
-            null,
-            null,
-            () -> 0L,
-            null,
-            null,
-            () -> true,
-            null,
-            emptyMap()
+            new NamedWriteableRegistry(Collections.emptyList())
         );
 
         assertTrue(context.indexSortedOnField("sort_field"));
@@ -445,6 +429,7 @@ public class SearchExecutionContextTests extends ESTestCase {
             0,
             0,
             indexSettings,
+            ClusterSettings.createBuiltInClusterSettings(),
             null,
             (mappedFieldType, fdc) -> mappedFieldType.fielddataBuilder(fdc).build(null, null),
             mapperService,

server/src/test/java/org/elasticsearch/search/aggregations/metrics/PercentilesTests.java

@@ -118,6 +118,6 @@ public class PercentilesTests extends BaseAggregationTestCase<PercentilesAggrega
         assertEquals(PercentilesMethod.TDIGEST, parsed.percentilesConfig().getMethod());
         var tdigestConfig = (PercentilesConfig.TDigest) parsed.percentilesConfig();
         assertEquals(200.0, tdigestConfig.getCompression(), 0);
-        assertEquals(TDigestExecutionHint.HIGH_ACCURACY, tdigestConfig.getExecutionHint());
+        assertEquals(TDigestExecutionHint.HIGH_ACCURACY, tdigestConfig.getExecutionHint(null));
     }
 }

server/src/test/java/org/elasticsearch/search/fetch/subphase/FieldFetcherTests.java

@@ -16,6 +16,7 @@ import org.elasticsearch.cluster.metadata.IndexMetadata;
 import org.elasticsearch.common.Strings;
 import org.elasticsearch.common.bytes.BytesReference;
 import org.elasticsearch.common.document.DocumentField;
+import org.elasticsearch.common.settings.ClusterSettings;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.index.IndexSettings;
 import org.elasticsearch.index.IndexVersion;
@@ -1265,6 +1266,7 @@ public class FieldFetcherTests extends MapperServiceTestCase {
             0,
             0,
             indexSettings,
+            ClusterSettings.createBuiltInClusterSettings(),
             null,
             indexFieldDataLookup,
             mapperService,

server/src/test/java/org/elasticsearch/search/fetch/subphase/highlight/HighlightBuilderTests.java

@@ -15,6 +15,7 @@ import org.elasticsearch.common.ParsingException;
 import org.elasticsearch.common.io.stream.BytesStreamOutput;
 import org.elasticsearch.common.io.stream.NamedWriteableRegistry;
 import org.elasticsearch.common.io.stream.StreamInput;
+import org.elasticsearch.common.settings.ClusterSettings;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.common.util.Maps;
 import org.elasticsearch.common.util.set.Sets;
@@ -298,6 +299,7 @@ public class HighlightBuilderTests extends ESTestCase {
             0,
             0,
             idxSettings,
+            ClusterSettings.createBuiltInClusterSettings(),
             null,
             null,
             null,

server/src/test/java/org/elasticsearch/search/rescore/QueryRescorerBuilderTests.java

@@ -14,6 +14,7 @@ import org.elasticsearch.Version;
 import org.elasticsearch.cluster.metadata.IndexMetadata;
 import org.elasticsearch.common.ParsingException;
 import org.elasticsearch.common.io.stream.NamedWriteableRegistry;
+import org.elasticsearch.common.settings.ClusterSettings;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.index.IndexSettings;
 import org.elasticsearch.index.mapper.MappedFieldType;
@@ -136,6 +137,7 @@ public class QueryRescorerBuilderTests extends ESTestCase {
             0,
             0,
             idxSettings,
+            ClusterSettings.createBuiltInClusterSettings(),
             null,
             null,
             null,
@@ -198,6 +200,7 @@ public class QueryRescorerBuilderTests extends ESTestCase {
             0,
             0,
             idxSettings,
+            ClusterSettings.createBuiltInClusterSettings(),
             null,
             null,
             null,

server/src/test/java/org/elasticsearch/search/sort/AbstractSortTestCase.java

@@ -12,6 +12,7 @@ import org.apache.lucene.search.IndexSearcher;
 import org.apache.lucene.search.SortField;
 import org.elasticsearch.cluster.metadata.IndexMetadata;
 import org.elasticsearch.common.io.stream.NamedWriteableRegistry;
+import org.elasticsearch.common.settings.ClusterSettings;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.env.Environment;
 import org.elasticsearch.index.Index;
@@ -200,6 +201,7 @@ public abstract class AbstractSortTestCase<T extends SortBuilder<T>> extends EST
             0,
             0,
             idxSettings,
+            ClusterSettings.createBuiltInClusterSettings(),
             bitsetFilterCache,
             indexFieldDataLookup,
             null,

server/src/test/java/org/elasticsearch/search/suggest/AbstractSuggestionBuilderTestCase.java

@@ -13,6 +13,7 @@ import org.elasticsearch.Version;
 import org.elasticsearch.cluster.metadata.IndexMetadata;
 import org.elasticsearch.common.io.stream.NamedWriteableRegistry;
 import org.elasticsearch.common.io.stream.Writeable;
+import org.elasticsearch.common.settings.ClusterSettings;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.index.Index;
 import org.elasticsearch.index.IndexSettings;
@@ -172,6 +173,7 @@ public abstract class AbstractSuggestionBuilderTestCase<SB extends SuggestionBui
                 0,
                 0,
                 idxSettings,
+                ClusterSettings.createBuiltInClusterSettings(),
                 null,
                 null,
                 mapperService,
@@ -228,6 +230,7 @@ public abstract class AbstractSuggestionBuilderTestCase<SB extends SuggestionBui
             0,
             0,
             idxSettings,
+            ClusterSettings.createBuiltInClusterSettings(),
             null,
             null,
             mock(MapperService.class),

test/framework/src/main/java/org/elasticsearch/index/mapper/MapperServiceTestCase.java

@@ -28,6 +28,7 @@ import org.elasticsearch.common.breaker.CircuitBreaker;
 import org.elasticsearch.common.bytes.BytesArray;
 import org.elasticsearch.common.bytes.BytesReference;
 import org.elasticsearch.common.compress.CompressedXContent;
+import org.elasticsearch.common.settings.ClusterSettings;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.common.util.BigArrays;
 import org.elasticsearch.common.util.MockBigArrays;
@@ -428,6 +429,11 @@ public abstract class MapperServiceTestCase extends ESTestCase {
                 throw new UnsupportedOperationException();
             }
 
+            @Override
+            public ClusterSettings getClusterSettings() {
+                throw new UnsupportedOperationException();
+            }
+
             @Override
             public MappedFieldType getFieldType(String path) {
                 return mapperService.fieldType(path);
@@ -630,17 +636,22 @@ public abstract class MapperServiceTestCase extends ESTestCase {
         IndexSettings indexSettings = new IndexSettings(indexMetadata, Settings.EMPTY);
         final SimilarityService similarityService = new SimilarityService(indexSettings, null, Map.of());
         final long nowInMillis = randomNonNegativeLong();
-        return new SearchExecutionContext(0, 0, indexSettings, new BitsetFilterCache(indexSettings, new BitsetFilterCache.Listener() {
-            @Override
-            public void onCache(ShardId shardId, Accountable accountable) {
+        return new SearchExecutionContext(
+            0,
+            0,
+            indexSettings,
+            ClusterSettings.createBuiltInClusterSettings(),
+            new BitsetFilterCache(indexSettings, new BitsetFilterCache.Listener() {
+                @Override
+                public void onCache(ShardId shardId, Accountable accountable) {
 
-            }
+                }
 
-            @Override
-            public void onRemoval(ShardId shardId, Accountable accountable) {
+                @Override
+                public void onRemoval(ShardId shardId, Accountable accountable) {
 
-            }
-        }),
+                }
+            }),
             (ft, fdc) -> ft.fielddataBuilder(fdc).build(new IndexFieldDataCache.None(), new NoneCircuitBreakerService()),
             mapperService,
             mapperService.mappingLookup(),

test/framework/src/main/java/org/elasticsearch/index/query/SearchExecutionContextHelper.java

@@ -0,0 +1,54 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0 and the Server Side Public License, v 1; you may not use this file except
+ * in compliance with, at your election, the Elastic License 2.0 or the Server
+ * Side Public License, v 1.
+ */
+
+package org.elasticsearch.index.query;
+
+import org.elasticsearch.common.io.stream.NamedWriteableRegistry;
+import org.elasticsearch.common.settings.ClusterSettings;
+import org.elasticsearch.index.IndexSettings;
+import org.elasticsearch.index.mapper.MappingLookup;
+import org.elasticsearch.xcontent.XContentParserConfiguration;
+
+import java.util.Collections;
+
+public class SearchExecutionContextHelper {
+    /**
+     * Factory method generating a dummy context with partial functionality.
+     */
+    public static SearchExecutionContext createSimple(
+        IndexSettings indexSettings,
+        XContentParserConfiguration parserConfiguration,
+        NamedWriteableRegistry namedWriteableRegistry
+    ) {
+        return new SearchExecutionContext(
+            0,
+            0,
+            indexSettings,
+            ClusterSettings.createBuiltInClusterSettings(),
+            null,
+            null,
+            null,
+            MappingLookup.EMPTY,
+            null,
+            null,
+            parserConfiguration,
+            namedWriteableRegistry,
+            null,
+            null,
+            System::currentTimeMillis,
+            null,
+            null,
+            () -> true,
+            null,
+            Collections.emptyMap()
+        );
+    }
+
+    // Allow no instances of this class.
+    private SearchExecutionContextHelper() {}
+}

test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java

@@ -55,6 +55,7 @@ import org.elasticsearch.common.io.stream.StreamOutput;
 import org.elasticsearch.common.lucene.Lucene;
 import org.elasticsearch.common.lucene.index.ElasticsearchDirectoryReader;
 import org.elasticsearch.common.network.NetworkAddress;
+import org.elasticsearch.common.settings.ClusterSettings;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.common.util.BigArrays;
 import org.elasticsearch.common.util.MockBigArrays;
@@ -340,6 +341,7 @@ public abstract class AggregatorTestCase extends ESTestCase {
             0,
             -1,
             indexSettings,
+            ClusterSettings.createBuiltInClusterSettings(),
             bitsetFilterCache,
             fieldDataBuilder,
             null,

test/framework/src/main/java/org/elasticsearch/test/AbstractBuilderTestCase.java

@@ -321,7 +321,7 @@ public abstract class AbstractBuilderTestCase extends ESTestCase {
      * @return a new {@link SearchExecutionContext} with the provided searcher
      */
     protected static SearchExecutionContext createSearchExecutionContext(IndexSearcher searcher) {
-        return serviceHolder.createShardContext(searcher);
+        return serviceHolder.createShardContext(searcher, null);
     }
 
     protected static CoordinatorRewriteContext createCoordinatorRewriteContext(
@@ -336,7 +336,7 @@ public abstract class AbstractBuilderTestCase extends ESTestCase {
      * @return a new {@link SearchExecutionContext} based on an index with no type registered
      */
     protected static SearchExecutionContext createShardContextWithNoType() {
-        return serviceHolderWithNoType.createShardContext(null);
+        return serviceHolderWithNoType.createShardContext(null, null);
     }
 
     /**
@@ -462,7 +462,7 @@ public abstract class AbstractBuilderTestCase extends ESTestCase {
                 parserConfiguration,
                 similarityService,
                 mapperRegistry,
-                () -> createShardContext(null),
+                () -> createShardContext(null, clusterService.getClusterSettings()),
                 idxSettings.getMode().idFieldMapperWithoutFieldData(),
                 ScriptCompiler.NONE
             );
@@ -553,11 +553,12 @@ public abstract class AbstractBuilderTestCase extends ESTestCase {
         @Override
         public void close() throws IOException {}
 
-        SearchExecutionContext createShardContext(IndexSearcher searcher) {
+        SearchExecutionContext createShardContext(IndexSearcher searcher, ClusterSettings clusterSettings) {
             return new SearchExecutionContext(
                 0,
                 0,
                 idxSettings,
+                clusterSettings == null ? ClusterSettings.createBuiltInClusterSettings() : clusterSettings,
                 bitsetFilterCache,
                 indexFieldDataService::getForField,
                 mapperService,

x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/AnalyticsAggregatorFactory.java

@@ -41,7 +41,7 @@ public class AnalyticsAggregatorFactory {
             (name, config, context, parent, percents, percentilesConfig, keyed, formatter, metadata) -> {
                 if (percentilesConfig.getMethod().equals(PercentilesMethod.TDIGEST)) {
                     double compression = ((PercentilesConfig.TDigest) percentilesConfig).getCompression();
-                    TDigestExecutionHint executionHint = ((PercentilesConfig.TDigest) percentilesConfig).getExecutionHint();
+                    TDigestExecutionHint executionHint = ((PercentilesConfig.TDigest) percentilesConfig).getExecutionHint(context);
                     return new HistoBackedTDigestPercentilesAggregator(
                         name,
                         config,
@@ -85,7 +85,7 @@ public class AnalyticsAggregatorFactory {
             (name, config, context, parent, percents, percentilesConfig, keyed, formatter, metadata) -> {
                 if (percentilesConfig.getMethod().equals(PercentilesMethod.TDIGEST)) {
                     double compression = ((PercentilesConfig.TDigest) percentilesConfig).getCompression();
-                    TDigestExecutionHint executionHint = ((PercentilesConfig.TDigest) percentilesConfig).getExecutionHint();
+                    TDigestExecutionHint executionHint = ((PercentilesConfig.TDigest) percentilesConfig).getExecutionHint(context);
                     return new HistoBackedTDigestPercentileRanksAggregator(
                         name,
                         config,

x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregationBuilder.java

@@ -51,7 +51,7 @@ public class BoxplotAggregationBuilder extends ValuesSourceAggregationBuilder.Me
     }
 
     private double compression = 100.0;
-    private TDigestExecutionHint executionHint = TDigestExecutionHint.DEFAULT;
+    private TDigestExecutionHint executionHint;
 
     public BoxplotAggregationBuilder(String name) {
         super(name);
@@ -82,9 +82,11 @@ public class BoxplotAggregationBuilder extends ValuesSourceAggregationBuilder.Me
     public BoxplotAggregationBuilder(StreamInput in) throws IOException {
         super(in);
         compression = in.readDouble();
-        executionHint = in.getTransportVersion().onOrAfter(TransportVersion.V_8_500_014)
-            ? TDigestExecutionHint.readFrom(in)
-            : TDigestExecutionHint.HIGH_ACCURACY;
+        if (in.getTransportVersion().onOrAfter(TransportVersion.V_8_500_018)) {
+            executionHint = in.readOptionalWriteable(TDigestExecutionHint::readFrom);
+        } else {
+            executionHint = TDigestExecutionHint.HIGH_ACCURACY;
+        }
     }
 
     @Override
@@ -95,8 +97,8 @@ public class BoxplotAggregationBuilder extends ValuesSourceAggregationBuilder.Me
     @Override
     protected void innerWriteTo(StreamOutput out) throws IOException {
         out.writeDouble(compression);
-        if (out.getTransportVersion().onOrAfter(TransportVersion.V_8_500_014)) {
-            executionHint.writeTo(out);
+        if (out.getTransportVersion().onOrAfter(TransportVersion.V_8_500_018)) {
+            out.writeOptionalWriteable(executionHint);
         }
     }
 
@@ -140,7 +142,9 @@ public class BoxplotAggregationBuilder extends ValuesSourceAggregationBuilder.Me
         AggregatorFactories.Builder subFactoriesBuilder
     ) throws IOException {
         BoxplotAggregatorSupplier aggregatorSupplier = context.getValuesSourceRegistry().getAggregator(REGISTRY_KEY, config);
-
+        if (executionHint == null) {
+            executionHint = TDigestExecutionHint.parse(context.getClusterSettings().get(TDigestExecutionHint.SETTING));
+        }
         return new BoxplotAggregatorFactory(
             name,
             config,
@@ -157,7 +161,9 @@ public class BoxplotAggregationBuilder extends ValuesSourceAggregationBuilder.Me
     @Override
     public XContentBuilder doXContentBody(XContentBuilder builder, Params params) throws IOException {
         builder.field(COMPRESSION_FIELD.getPreferredName(), compression);
-        builder.field(EXECUTION_HINT_FIELD.getPreferredName(), executionHint);
+        if (executionHint != null) {
+            builder.field(EXECUTION_HINT_FIELD.getPreferredName(), executionHint);
+        }
         return builder;
     }
 
@@ -167,7 +173,7 @@ public class BoxplotAggregationBuilder extends ValuesSourceAggregationBuilder.Me
         if (obj == null || getClass() != obj.getClass()) return false;
         if (super.equals(obj) == false) return false;
         BoxplotAggregationBuilder other = (BoxplotAggregationBuilder) obj;
-        return Objects.equals(compression, other.compression) && executionHint.equals(other.executionHint);
+        return Objects.equals(compression, other.compression) && Objects.equals(executionHint, other.executionHint);
     }
 
     @Override

x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/cluster/routing/allocation/mapper/DataTierFieldTypeTests.java

@@ -17,10 +17,10 @@ import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.index.IndexSettings;
 import org.elasticsearch.index.mapper.MappedFieldType;
 import org.elasticsearch.index.mapper.MapperServiceTestCase;
-import org.elasticsearch.index.mapper.MappingLookup;
 import org.elasticsearch.index.mapper.ValueFetcher;
 import org.elasticsearch.index.query.QueryShardException;
 import org.elasticsearch.index.query.SearchExecutionContext;
+import org.elasticsearch.index.query.SearchExecutionContextHelper;
 import org.elasticsearch.search.lookup.Source;
 import org.elasticsearch.xcontent.XContentType;
 
@@ -30,7 +30,6 @@ import java.util.Arrays;
 import java.util.List;
 import java.util.function.Predicate;
 
-import static java.util.Collections.emptyMap;
 import static java.util.Collections.singletonList;
 import static org.hamcrest.Matchers.containsString;
 
@@ -114,27 +113,7 @@ public class DataTierFieldTypeTests extends MapperServiceTestCase {
         IndexSettings indexSettings = new IndexSettings(indexMetadata, Settings.EMPTY);
 
         Predicate<String> indexNameMatcher = pattern -> Regex.simpleMatch(pattern, "index");
-        return new SearchExecutionContext(
-            0,
-            0,
-            indexSettings,
-            null,
-            null,
-            null,
-            MappingLookup.EMPTY,
-            null,
-            null,
-            parserConfig(),
-            writableRegistry(),
-            null,
-            null,
-            System::currentTimeMillis,
-            null,
-            indexNameMatcher,
-            () -> true,
-            null,
-            emptyMap()
-        );
+        return SearchExecutionContextHelper.createSimple(indexSettings, parserConfig(), writableRegistry());
     }
 
     private SearchExecutionContext createContextWithoutSetting() {
@@ -144,26 +123,6 @@ public class DataTierFieldTypeTests extends MapperServiceTestCase {
             .numberOfReplicas(0)
             .build();
         IndexSettings indexSettings = new IndexSettings(indexMetadata, Settings.EMPTY);
-        return new SearchExecutionContext(
-            0,
-            0,
-            indexSettings,
-            null,
-            null,
-            null,
-            MappingLookup.EMPTY,
-            null,
-            null,
-            parserConfig(),
-            writableRegistry(),
-            null,
-            null,
-            System::currentTimeMillis,
-            null,
-            value -> true,
-            () -> true,
-            null,
-            emptyMap()
-        );
+        return SearchExecutionContextHelper.createSimple(indexSettings, parserConfig(), writableRegistry());
     }
 }

x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/accesscontrol/DocumentSubsetBitsetCacheTests.java

@@ -31,6 +31,7 @@ import org.apache.lucene.util.BitSet;
 import org.elasticsearch.client.internal.Client;
 import org.elasticsearch.common.CheckedBiConsumer;
 import org.elasticsearch.common.logging.Loggers;
+import org.elasticsearch.common.settings.ClusterSettings;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.core.CheckedConsumer;
 import org.elasticsearch.index.IndexSettings;
@@ -602,6 +603,7 @@ public class DocumentSubsetBitsetCacheTests extends ESTestCase {
                 shardId.id(),
                 0,
                 indexSettings,
+                ClusterSettings.createBuiltInClusterSettings(),
                 null,
                 null,
                 null,

x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/accesscontrol/SecurityIndexReaderWrapperIntegrationTests.java

@@ -26,6 +26,7 @@ import org.elasticsearch.client.internal.Client;
 import org.elasticsearch.common.bytes.BytesArray;
 import org.elasticsearch.common.bytes.BytesReference;
 import org.elasticsearch.common.lucene.index.ElasticsearchDirectoryReader;
+import org.elasticsearch.common.settings.ClusterSettings;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.common.util.concurrent.ThreadContext;
 import org.elasticsearch.index.IndexSettings;
@@ -89,6 +90,7 @@ public class SecurityIndexReaderWrapperIntegrationTests extends AbstractBuilderT
             shardId.id(),
             0,
             indexSettings,
+            ClusterSettings.createBuiltInClusterSettings(),
             null,
             null,
             null,
@@ -246,6 +248,7 @@ public class SecurityIndexReaderWrapperIntegrationTests extends AbstractBuilderT
             shardId.id(),
             0,
             indexSettings,
+            ClusterSettings.createBuiltInClusterSettings(),
             null,
             null,
             null,

x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/job/RollupIndexerIndexingTests.java

@@ -39,11 +39,11 @@ import org.elasticsearch.index.mapper.DateFieldMapper;
 import org.elasticsearch.index.mapper.KeywordFieldMapper;
 import org.elasticsearch.index.mapper.MappedFieldType;
 import org.elasticsearch.index.mapper.MapperBuilderContext;
-import org.elasticsearch.index.mapper.MappingLookup;
 import org.elasticsearch.index.mapper.NumberFieldMapper;
 import org.elasticsearch.index.mapper.NumberFieldMapper.NumberType;
 import org.elasticsearch.index.query.RangeQueryBuilder;
 import org.elasticsearch.index.query.SearchExecutionContext;
+import org.elasticsearch.index.query.SearchExecutionContextHelper;
 import org.elasticsearch.script.ScriptCompiler;
 import org.elasticsearch.search.aggregations.Aggregations;
 import org.elasticsearch.search.aggregations.AggregatorTestCase;
@@ -80,7 +80,6 @@ import java.util.concurrent.TimeUnit;
 import java.util.concurrent.atomic.AtomicReference;
 import java.util.function.Consumer;
 
-import static java.util.Collections.emptyMap;
 import static org.hamcrest.Matchers.equalTo;
 import static org.hamcrest.core.IsInstanceOf.instanceOf;
 import static org.hamcrest.number.OrderingComparison.greaterThan;
@@ -92,27 +91,7 @@ public class RollupIndexerIndexingTests extends AggregatorTestCase {
     @Before
     private void setup() {
         settings = createIndexSettings();
-        searchExecutionContext = new SearchExecutionContext(
-            0,
-            0,
-            settings,
-            null,
-            null,
-            null,
-            MappingLookup.EMPTY,
-            null,
-            null,
-            null,
-            null,
-            null,
-            null,
-            () -> 0L,
-            null,
-            null,
-            () -> true,
-            null,
-            emptyMap()
-        );
+        searchExecutionContext = SearchExecutionContextHelper.createSimple(settings, null, null);
     }
 
     public void testSimpleDateHisto() throws Exception {

x-pack/plugin/wildcard/src/test/java/org/elasticsearch/xpack/wildcard/mapper/WildcardFieldMapperTests.java

@@ -45,6 +45,7 @@ import org.apache.lucene.util.automaton.RegExp;
 import org.elasticsearch.Version;
 import org.elasticsearch.cluster.metadata.IndexMetadata;
 import org.elasticsearch.common.lucene.search.AutomatonQueries;
+import org.elasticsearch.common.settings.ClusterSettings;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.common.unit.Fuzziness;
 import org.elasticsearch.core.Tuple;
@@ -1094,6 +1095,7 @@ public class WildcardFieldMapperTests extends MapperTestCase {
             0,
             0,
             idxSettings,
+            ClusterSettings.createBuiltInClusterSettings(),
             bitsetFilterCache,
             indexFieldDataLookup,
             null,