|
@@ -55,7 +55,14 @@ In many cases, a brute-force kNN search is not efficient enough. For this reason
|
|
|
|
|
|
Unmapped array fields of float elements with size between 128 and 4096 are dynamically mapped as `dense_vector` with a default similariy of `cosine`. You can override the default similarity by explicitly mapping the field as `dense_vector` with the desired similarity.
|
|
|
|
|
|
-Indexing is enabled by default for dense vector fields and indexed as `bbq_hnsw` if dimensions are greater than or equal to 384, otherwise they are indexed as `int8_hnsw`. When indexing is enabled, you can define the vector similarity to use in kNN search:
|
|
|
+Indexing is enabled by default for dense vector fields and indexed as `bbq_hnsw` if dimensions are greater than or equal to 384, otherwise they are indexed as `int8_hnsw`. {applies_to}`stack: ga 9.1`
|
|
|
+
|
|
|
+:::{note}
|
|
|
+In {{stack}} 9.0, dense vector fields are always indexed as `int8_hnsw`.
|
|
|
+:::
|
|
|
+
|
|
|
+
|
|
|
+When indexing is enabled, you can define the vector similarity to use in kNN search:
|
|
|
|
|
|
```console
|
|
|
PUT my-index-2
|
|
@@ -107,6 +114,10 @@ When using a quantized format, you may want to oversample and rescore the result
|
|
|
|
|
|
To use a quantized index, you can set your index type to `int8_hnsw`, `int4_hnsw`, or `bbq_hnsw`. When indexing `float` vectors, the current default index type is `bbq_hnsw` for vectors with greater than or equal to 384 dimensions, otherwise it's `int8_hnsw`.
|
|
|
|
|
|
+:::{note}
|
|
|
+In {{stack}} 9.0, dense vector fields are always indexed as `int8_hnsw`.
|
|
|
+:::
|
|
|
+
|
|
|
Quantized vectors can use [oversampling and rescoring](docs-content://solutions/search/vector/knn.md#dense-vector-knn-search-rescoring) to improve accuracy on approximate kNN search results.
|
|
|
|
|
|
::::{note}
|
|
@@ -255,9 +266,16 @@ $$$dense-vector-index-options$$$
|
|
|
`type`
|
|
|
: (Required, string) The type of kNN algorithm to use. Can be either any of:
|
|
|
* `hnsw` - This utilizes the [HNSW algorithm](https://arxiv.org/abs/1603.09320) for scalable approximate kNN search. This supports all `element_type` values.
|
|
|
- * `int8_hnsw` - The default index type for float vectors with less than 384 dimensions. This utilizes the [HNSW algorithm](https://arxiv.org/abs/1603.09320) in addition to automatically scalar quantization for scalable approximate kNN search with `element_type` of `float`. This can reduce the memory footprint by 4x at the cost of some accuracy. See [Automatically quantize vectors for kNN search](#dense-vector-quantization).
|
|
|
+ * `int8_hnsw` - The default index type for some float vectors:
|
|
|
+
|
|
|
+ * {applies_to}`stack: ga 9.1` Default for float vectors with less than 384 dimensions.
|
|
|
+ * {applies_to}`stack: ga 9.0` Default for float all vectors.
|
|
|
+
|
|
|
+ This utilizes the [HNSW algorithm](https://arxiv.org/abs/1603.09320) in addition to automatically scalar quantization for scalable approximate kNN search with `element_type` of `float`. This can reduce the memory footprint by 4x at the cost of some accuracy. See [Automatically quantize vectors for kNN search](#dense-vector-quantization).
|
|
|
* `int4_hnsw` - This utilizes the [HNSW algorithm](https://arxiv.org/abs/1603.09320) in addition to automatically scalar quantization for scalable approximate kNN search with `element_type` of `float`. This can reduce the memory footprint by 8x at the cost of some accuracy. See [Automatically quantize vectors for kNN search](#dense-vector-quantization).
|
|
|
- * `bbq_hnsw` - The default index type for float vectors with greater than or equal to 384 dimensions. This utilizes the [HNSW algorithm](https://arxiv.org/abs/1603.09320) in addition to automatically binary quantization for scalable approximate kNN search with `element_type` of `float`. This can reduce the memory footprint by 32x at the cost of accuracy. See [Automatically quantize vectors for kNN search](#dense-vector-quantization).
|
|
|
+ * `bbq_hnsw` - This utilizes the [HNSW algorithm](https://arxiv.org/abs/1603.09320) in addition to automatically binary quantization for scalable approximate kNN search with `element_type` of `float`. This can reduce the memory footprint by 32x at the cost of accuracy. See [Automatically quantize vectors for kNN search](#dense-vector-quantization).
|
|
|
+
|
|
|
+ {applies_to}`stack: ga 9.1` `bbq_hnsw` is the default index type for float vectors with greater than or equal to 384 dimensions.
|
|
|
* `flat` - This utilizes a brute-force search algorithm for exact kNN search. This supports all `element_type` values.
|
|
|
* `int8_flat` - This utilizes a brute-force search algorithm in addition to automatically scalar quantization. Only supports `element_type` of `float`.
|
|
|
* `int4_flat` - This utilizes a brute-force search algorithm in addition to automatically half-byte scalar quantization. Only supports `element_type` of `float`.
|