elasticsearch/docs/README.asciidoc

= Elasticsearch docs

This is a guide for writing, editing, and building the following Elasticsearch
docs:

- https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html[Elasticsearch Guide]
- https://www.elastic.co/guide/en/elasticsearch/painless/current/index.html[Painless Scripting Language]
- https://www.elastic.co/guide/en/elasticsearch/plugins/current/index.html[Elasticsearch Plugins and Integrations]
- https://www.elastic.co/guide/en/elasticsearch/resiliency/current/index.html[Elasticsearch Resiliency Status]
- https://www.elastic.co/guide/en/elasticsearch/client/community/current/index.html[Community Contributed Clients]
- https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/index.html[Java REST Client]

== Build the docs

The Elasticsearch docs consist of `.asciidoc` files, written in the
https://docs.asciidoctor.org/asciidoc/latest/[Asciidoctor] flavor. You can build
the docs locally using the steps in the
https://github.com/elastic/docs#readme[elastic/docs README].

=== Preview links for PRs

For open PRs, a Buildkite job called `docs-build-pr` generates a live preview of any
docs changes. If the check runs successfully, you can find the preview at:

`https://elasticsearch_bk_<PR_NUM>.docs-preview.app.elstc.co/diff`

`docs-build-pr` runs automatically for PRs opened by Elastic employees.
To run CI checks on PRs from external contributors, an Elastic employee can
leave a GitHub comment containing `run docs-build`.

To re-run `docs-build-pr`, an Elastic employee can leave a GitHub
comment containing `run docs-build`.

== Add API reference pages

When you need to add a reference page for a new API:

* Use the API reference template: 
https://github.com/elastic/docs/blob/master/shared/api-ref-ex.asciidoc[template], 
https://www.elastic.co/guide/en/elasticsearch/reference/master/get-snapshot-repo-api.html[real-life example].
* Create a separate source file for each API. 
* The anchor for the top-level heading should match the heading as closely as possible. 
* Use dashes to separate words in the anchor. (Avoid underscores.) 
* The name of the source file should match the anchor text. 
* Include an abbreviated title that drops the heading's _API_ suffix.
* Start the short description with a verb.
* Include the sections from the reference template as applicable. 
See the template for information about each section. 
* Add a link to the new topic from the appropriate API category page.
These links are typically grouped into sub-categories and listed in logical order. 
For example, the put API is listed before the get and delete APIs.
* Include the new file so the API appears in alphabetical order in the TOC nav.

The template uses https://github.com/elastic/docs/blob/master/shared/attributes.asciidoc[shared attributes] 
for a number of the standard headings so we can tweak them globally if need be. 

Many of the API reference pages also use shared parameter definitions from 
https://github.com/elastic/elasticsearch/blob/master/docs/reference/rest-api/common-parms.asciidoc[elasticsearch/docs/reference/rest-api/common-parms].
You don't have to do that. 

== Backport doc fixes

* Doc changes should generally be made against master and backported through to the current version
  (as applicable).  
  
* Changes can also be backported to the maintenance version of the previous major version. 
  This is typically reserved for technical corrections, as it can require resolving more complex
  merge conflicts, fixing test failures, and figuring out where to apply the change.

* Avoid backporting to out-of-maintenance versions. 
  Docs follow the same policy as code and fixes are not ordinarily merged to
  versions that are out of maintenance.
  
* Do not backport doc changes to https://www.elastic.co/support/eol[EOL versions].

* Release notes for known issues are an exception to this policy. Document the 
  known issue in the release notes for any minor version affected by the issue.
  Backport the changes to any branch containing release notes for those
  versions, even if the branch is no longer maintained.

== Test code snippets

Snippets marked with `[source,console]` are automatically annotated with
"VIEW IN CONSOLE" and "COPY AS CURL" in the documentation and are automatically
tested by the command `./gradlew -pdocs check`. To test just the docs from a
single page, use e.g. `./gradlew -pdocs yamlRestTest --tests "\*rollover*"`.

By default each `[source,console]` snippet runs as its own isolated test. You
can manipulate the test execution in the following ways:

* `// TEST`: Explicitly marks a snippet as a test. Snippets marked this way
are tests even if they don't have `[source,console]` but usually `// TEST` is
used for its modifiers:
  * `// TEST[s/foo/bar/]`: Replace `foo` with `bar` in the generated test. This
  should be used sparingly because it makes the snippet "lie". Sometimes,
  though, you can use it to make the snippet more clear. Keep in mind that
  if there are multiple substitutions then they are applied in the order that
  they are defined.
  * `// TEST[catch:foo]`: Used to expect errors in the requests. Replace `foo`
  with `request` to expect a 400 error, for example. If the snippet contains
  multiple requests then only the last request will expect the error.
  * `// TEST[continued]`: Continue the test started in the last snippet. Between
  tests the nodes are cleaned: indexes are removed, etc. This prevents that
  from happening between snippets because the two snippets are a single test.
  This is most useful when you have text and snippets that work together to
  tell the story of some use case because it merges the snippets (and thus the
  use case) into one big test.
      * You can't use `// TEST[continued]` immediately after `// TESTSETUP` or
      `// TEARDOWN`.
  * `// TEST[skip:reason]`: Skip this test. Replace `reason` with the actual
  reason to skip the test. Snippets without `// TEST` or `// CONSOLE` aren't
  considered tests anyway but this is useful for explicitly documenting the
  reason why the test shouldn't be run.
  * `// TEST[setup:name]`: Run some setup code before running the snippet. This
  is useful for creating and populating indexes used in the snippet. The `name`
  is split on `,` and looked up in the `setups` defined in `docs/build.gradle`.
  See `// TESTSETUP` below for a similar feature.
  * `// TEST[teardown:name]`: Run some teardown code after the snippet.
  This is useful for performing hidden cleanup, such as deleting index templates. The
  `name` is split on `,` and looked up in the `teardowns` defined in
  `docs/build.gradle`. See `// TESTSETUP` below for a similar feature.
  * `// TEST[warning:some warning]`: Expect the response to include a `Warning`
  header. If the response doesn't include a `Warning` header with the exact
  text then the test fails. If the response includes `Warning` headers that
  aren't expected then the test fails.
* `[source,console-result]`: Matches this snippet against the body of the
  response of the last test. If the response is JSON then order is ignored. If
  you add `// TEST[continued]` to the snippet after `[source,console-result]`
  it will continue in the same test, allowing you to interleave requests with
  responses to check.
* `// TESTRESPONSE`: Explicitly marks a snippet as a test response even without
  `[source,console-result]`. Similarly to `// TEST` this is mostly used for
  its modifiers.
  * You can't use `[source,console-result]` immediately after `// TESTSETUP`.
  Instead, consider using `// TEST[continued]` or rearrange your snippets.

  NOTE: Previously we only used `// TESTRESPONSE` instead of
  `[source,console-result]` so you'll see that a lot in older branches but we
  prefer `[source,console-result]` now.

  * `// TESTRESPONSE[s/foo/bar/]`: Substitutions. See `// TEST[s/foo/bar]` for
  how it works. These are much more common than `// TEST[s/foo/bar]` because
  they are useful for eliding portions of the response that are not pertinent
  to the documentation.
    * One interesting difference here is that you often want to match against
    the response from Elasticsearch. To do that you can reference the "body" of
    the response like this: `// TESTRESPONSE[s/"took": 25/"took": $body.took/]`.
    Note the `$body` string. This says "I don't expect that 25 number in the
    response, just match against what is in the response." Instead of writing
    the path into the response after `$body` you can write `$_path` which
    "figures out" the path. This is especially useful for making sweeping
    assertions like "I made up all the numbers in this example, don't compare
    them" which looks like `// TESTRESPONSE[s/\d+/$body.$_path/]`.
  * `// TESTRESPONSE[non_json]`: Add substitutions for testing responses in a
  format other than JSON. Use this after all other substitutions so it doesn't
  make other substitutions difficult.
  * `// TESTRESPONSE[skip:reason]`: Skip the assertions specified by this
  response.
* `// TESTSETUP`: Marks this snippet as the "setup" for all other snippets in
  this file. In order to enhance clarity and simplify understanding for readers,
  a straightforward approach involves marking the first snippet in the documentation file with the 
  `// TESTSETUP` marker. By doing so, it clearly indicates that this particular snippet serves as the setup 
  or preparation step for all subsequent snippets in the file. 
  This helps in explaining the necessary steps that need to be executed before running the examples.
  Unlike the alternative convention `// TEST[setup:name]`, which relies on a setup defined in a separate file,
  this convention brings the setup directly into the documentation file, making it more self-contained and reducing ambiguity.
  By adopting this convention, users can easily identify and follow the correct sequence 
  of steps to ensure that the examples provided in the documentation work as intended.
* `// TEARDOWN`: Ends and cleans up a test series started with `// TESTSETUP` or
  `// TEST[setup:name]`. You can use `// TEARDOWN` to set up multiple tests in
  the same file.
* `// NOTCONSOLE`: Marks this snippet as neither `// CONSOLE` nor
  `// TESTRESPONSE`, excluding it from the list of unconverted snippets. We
  should only use this for snippets that *are* JSON but are *not* responses or
  requests.

In addition to the standard CONSOLE syntax these snippets can contain blocks
of yaml surrounded by markers like this:

```
startyaml
  - compare_analyzers: {index: thai_example, first: thai, second: rebuilt_thai}
endyaml
```

This allows slightly more expressive testing of the snippets. Since that syntax
is not supported by `[source,console]` the usual way to incorporate it is with a
`// TEST[s//]` marker like this:

```
// TEST[s/\n$/\nstartyaml\n  - compare_analyzers: {index: thai_example, first: thai, second: rebuilt_thai}\nendyaml\n/]
```

Any place you can use json you can use elements like `$body.path.to.thing`
which is replaced on the fly with the contents of the thing at `path.to.thing`
in the last response.