|
|
@@ -0,0 +1,88 @@
|
|
|
+[[archive-indices]]
|
|
|
+== Reading indices from older {es} versions
|
|
|
+
|
|
|
+{es} has full query and write support for indices created in the previous major
|
|
|
+version. If you have indices created in {es} versions 5 or 6, you can now use
|
|
|
+the archive functionality to import them into newer {es} versions as well.
|
|
|
+
|
|
|
+The archive functionality provides slower read-only access to older {es} data,
|
|
|
+for compliance or regulatory reasons, the occasional lookback or investigation,
|
|
|
+or to rehydrate parts of it. Access to the data is expected to be infrequent,
|
|
|
+and can therefore happen with limited performance and query capabilities.
|
|
|
+
|
|
|
+For this, {es} has the ability to access older snapshot repositories
|
|
|
+(going back to version 5). The legacy indices in the <<snapshot-restore,snapshot repository>>
|
|
|
+can either be <<restore-snapshot-api,restored>>, or can be directly accessed
|
|
|
+via <<searchable-snapshots,searchable snapshots>> so that the archived data
|
|
|
+won't even need to fully reside on local disks for access.
|
|
|
+
|
|
|
+[discrete]
|
|
|
+[[archive-indices-supported-field-types]]
|
|
|
+=== Supported field types
|
|
|
+
|
|
|
+Old mappings are imported as much "as-is" as possible into {es} 8, but only
|
|
|
+provide regular query capabilities on a select subset of fields:
|
|
|
+
|
|
|
+- <<number,Numeric types>>
|
|
|
+- <<boolean,`boolean` type>>
|
|
|
+- <<ip,`ip` type>>
|
|
|
+- <<geo-point,`geo_point` type>>
|
|
|
+- <<date,`date` types>>: the date `format` setting on date fields is supported
|
|
|
+ as long as it behaves similarly across these versions. In case it is not,
|
|
|
+ for example {ref-7x}/migrate-to-java-time.html[when using custom date formats],
|
|
|
+ this field can be updated on legacy indices so that it can be changed by a
|
|
|
+ user if need be.
|
|
|
+- <<keyword-field-type,`keyword` type>>: the `normalizer` setting on keyword
|
|
|
+ fields is supported as long as it behaves similarly across these versions.
|
|
|
+ In case it is not, this field can be updated on legacy indices if need be.
|
|
|
+- <<text-field-type,`text` type>>: scoring capabilities are limited, and all
|
|
|
+ queries return constant scores that are equal to 1.0. The `analyzer`
|
|
|
+ settings on text fields are supported as long as they behave similarly
|
|
|
+ across these versions. In case they do not, they can be updated on legacy
|
|
|
+ indices if need be.
|
|
|
+- <<multi-fields,Multi-fields>>
|
|
|
+- <<field-alias,Field aliases>>
|
|
|
+- <<object,`object`>> fields
|
|
|
+- some basic metadata fields, e.g. `_type` for querying {es} 5 indices
|
|
|
+- <<runtime-mapping-fields,runtime fields>>
|
|
|
+- <<mapping-source-field,`_source` field>>
|
|
|
+
|
|
|
+{es} 5 indices with mappings that have {ref-7x}/removal-of-types.html[multiple mapping types]
|
|
|
+are collapsed together on a best-effort basis before they are imported.
|
|
|
+
|
|
|
+In case the auto-import of mappings does not work, or the new {es} version
|
|
|
+can't make sense of the mapping, it falls back to importing the index without
|
|
|
+the mapping, but stores the original mapping in the <<mapping-meta-field,_meta>>
|
|
|
+section of the imported index. The legacy mapping can then be introspected
|
|
|
+using the <<indices-get-mapping,GET mapping>> API and an updated mapping can
|
|
|
+be manually put in place using the <<indices-put-mapping,update mapping>> API,
|
|
|
+copying and adapting relevant sections of the legacy mapping to work with the
|
|
|
+current {es} version. While auto-import is expected to work in most cases,
|
|
|
+failures of doing so should be https://github.com/elastic/elasticsearch/issues/new/choose[raised]
|
|
|
+with the Elastic team for future improvements.
|
|
|
+
|
|
|
+[discrete]
|
|
|
+=== Supported APIs
|
|
|
+
|
|
|
+Archive indices are read-only, and provide data access via the
|
|
|
+<<search-search,search>> and <<search-field-caps,field capabilities>> APIs.
|
|
|
+They do not support the <<docs-get,Get API>> nor any write APIs.
|
|
|
+
|
|
|
+Archive indices allow running queries as well as aggregations in so far as
|
|
|
+they are <<archive-indices-supported-field-types,supported by the given field type>>.
|
|
|
+
|
|
|
+Due to `_source` access the data can also be <<docs-reindex,reindexed>>
|
|
|
+to a new index that has full compatibility with the current {es} version.
|
|
|
+
|
|
|
+[discrete]
|
|
|
+=== How to upgrade older {es} 5 or 6 clusters?
|
|
|
+
|
|
|
+Take a snapshot of the indices in the old cluster, delete indices that are not
|
|
|
+directly supported by ES 8 (i.e. indices older than 7.0), upgrade the cluster
|
|
|
+without the old indices, and then <<restore-snapshot-api,restore>> the legacy
|
|
|
+indices from the snapshot or <<searchable-snapshots-api-mount-snapshot,mount>>
|
|
|
+them via searchable snapshots.
|
|
|
+
|
|
|
+In the future, we plan on streamlining the upgrade process going forward,
|
|
|
+making it easier to take legacy indices along when going to future major
|
|
|
+{es} versions.
|
Yannick Welsch