|
@@ -97,9 +97,6 @@ A `requires` section defines requirements that have to be met in order for tests
|
|
|
- `cluster_features` to <<cluster_features, require cluster features>>.
|
|
|
- `test_runner_features` to <<requires_test_runner_features, require test runner features>>.
|
|
|
|
|
|
-A `requires` section must specify at least one of the two options above.
|
|
|
-If `cluster_features` are required, a `reason` must be provided in addition.
|
|
|
-
|
|
|
A `skip` section, on the other hand, defines certain conditions that, if met, will skip the test, such as:
|
|
|
|
|
|
- `capabilities` to <<capabilities_check, skip if API capabilities are present>>.
|
|
@@ -110,10 +107,10 @@ A `skip` section, on the other hand, defines certain conditions that, if met, wi
|
|
|
- `features`: Only kept for a transition period, please use <<requires_test_runner_features, `test_runner_features`>>
|
|
|
in the `requires` section instead.
|
|
|
|
|
|
-A `skip` section must specify at least one of the options above.
|
|
|
-Unless only legacy test runner `features` are required, a `reason` must also be provided.
|
|
|
+`requires` and `skip` sections must specify at least one of the options mentioned above.
|
|
|
+Unless only `test_runner_features` or legacy test runner `features` are specified, a `reason` must be given.
|
|
|
|
|
|
-`requires` and / or `skip` can also be used at the top level of the file in the `setup` and `teardown` blocks,
|
|
|
+`requires` and `skip` can also be used at the top level of the file in the `setup` and `teardown` blocks,
|
|
|
so all the tests in a file will be skipped if either any requirement fails or any skip condition applies regardless
|
|
|
if defined in `setup` and `teardown`.
|
|
|
|
|
@@ -143,6 +140,8 @@ other test runners to skip tests if they do not support the capabilities API yet
|
|
|
... test definitions ...
|
|
|
....
|
|
|
|
|
|
+The `capabilities` field is an array containing one or several capabilities checks.
|
|
|
+
|
|
|
*NOTE: If planning to `skip` on capabilities, keep in mind this might lead to unexpected results in _mixed cluster_
|
|
|
tests!* A test is only skipped if *all* nodes support the requested capabilities, in _mixed clusters_ this might not be
|
|
|
the case: such a cluster can consist of a mix of nodes where some support respective capabilities and others don't,
|
|
@@ -168,13 +167,15 @@ public Set<String> supportedCapabilities() {
|
|
|
[[cluster_features]]
|
|
|
=== Require or skip cluster features
|
|
|
|
|
|
-In the past, Elasticsearch has been over-reliant on release version checks all across its code base.
|
|
|
-This has become a limiting factor when deploying Elasticsearch in a more continuous fashion.
|
|
|
+Cluster features indicate a particular high-level _internal_ functionality and are used for coordination within
|
|
|
+the Elasticsearch cluster to enable functionality once supported on all nodes, e.g. usage of a new transport endpoint.
|
|
|
|
|
|
-Moving forward, the goal is to base any such check on the availability of a cluster feature.
|
|
|
-Rather than <<skip_version, skipping tests based on various release version ranges>>, you can
|
|
|
-require `cluster_features` to be either present (`requires`) and / or absent (`skip`) in order to run a test.
|
|
|
-For instance:
|
|
|
+In contrast to <<capabilities_check, capabilities>>, cluster features are strictly internal, though can also be used for
|
|
|
+skipping REST tests. Cluster features are not meant to be extremely fine-grained. In case you are not sure if you need
|
|
|
+a cluster feature, <<capabilities_check, capabilities>> might be the better choice.
|
|
|
+
|
|
|
+To select applicable tests (e.g. in backwards compatibility or mixed cluster tests), you can require `cluster_features`
|
|
|
+to be either present (`requires`) or absent (`skip`), for instance:
|
|
|
|
|
|
....
|
|
|
"Parent":
|
|
@@ -218,6 +219,8 @@ introduced by the former feature and eventually fixed by the latter one. For ins
|
|
|
... test definitions ...
|
|
|
....
|
|
|
|
|
|
+The `known_issues` field is an array containing one or several issues.
|
|
|
+
|
|
|
Note: If a known issue cannot be defined in terms of existing cluster features, the previously described
|
|
|
<<synthetic_cluster_features,synthetic version based cluster features>> can be used.
|
|
|
|