[DOCS] Adds HuggingFace example to inference API tutorial (#107298)

docs/reference/inference/put-inference.asciidoc

@@ -346,7 +346,7 @@ Example response:
 ===== Hugging Face service
 
 The following example shows how to create an {infer} endpoint called
-`hugging-face_embeddings` to perform a `text_embedding` task type.
+`hugging-face-embeddings` to perform a `text_embedding` task type.
 
 [source,console]
 ------------------------------------------------------------
@@ -371,6 +371,19 @@ endpoint URL. Select the model you want to use on the new endpoint creation page
 task under the Advanced configuration section. Create the endpoint. Copy the URL
 after the endpoint initialization has been finished.
 
+[discrete]
+[[inference-example-hugging-face-supported-models]]
+The list of supported models for the Hugging Face service:
+
+* https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2[all-MiniLM-L6-v2]
+* https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2[all-MiniLM-L12-v2]
+* https://huggingface.co/sentence-transformers/all-mpnet-base-v2[all-mpnet-base-v2]
+* https://huggingface.co/intfloat/e5-base-v2[e5-base-v2]
+* https://huggingface.co/intfloat/e5-small-v2[e5-small-v2]
+* https://huggingface.co/intfloat/multilingual-e5-base[multilingual-e5-base]
+* https://huggingface.co/intfloat/multilingual-e5-small[multilingual-e5-small]
+
+
 [discrete]
 [[inference-example-eland]]
 ===== Models uploaded by Eland via the elasticsearch service

docs/reference/search/search-your-data/semantic-search-inference.asciidoc

@@ -6,9 +6,11 @@
 
 The instructions in this tutorial shows you how to use the {infer} API with
 various services to perform semantic search on your data. The following examples 
-use Cohere's `embed-english-v3.0` model and OpenAI's `text-embedding-ada-002`
-second generation embedding model. You can use any Cohere and OpenAI models,
-they are all supported by the {infer} API.
+use Cohere's `embed-english-v3.0` model, the `all-mpnet-base-v2` model from
+HuggingFace, and OpenAI's `text-embedding-ada-002` second generation embedding
+model. You can use any Cohere and OpenAI models, they are all supported by the
+{infer} API. For a list of supported models available on HuggingFace, refer to
+<<inference-example-hugging-face-supported-models, the supported model list>>.
 
 Click the name of the service you want to use on any of the widgets below to
 review the corresponding instructions.
@@ -91,7 +93,7 @@ GET _tasks/<task_id>
 // TEST[skip:TBD]
 
 You can also cancel the reindexing process if you don't want to wait until the
-reindexing process is fully complete which might take hours:
+reindexing process is fully complete which might take hours for large data sets:
 
 [source,console]
 ----
@@ -104,7 +106,7 @@ POST _tasks/<task_id>/_cancel
 [[infer-semantic-search]]
 ==== Semantic search
 
-After the dataset has been enriched with the embeddings, you can query the data
+After the data set has been enriched with the embeddings, you can query the data
 using {ref}/knn-search.html#knn-semantic-search[semantic search]. Pass a
 `query_vector_builder` to the k-nearest neighbor (kNN) vector search API, and
 provide the query text and the model you have used to create the embeddings.

docs/reference/tab-widgets/inference-api/infer-api-ingest-pipeline-widget.asciidoc

@@ -7,6 +7,12 @@
             id="infer-api-ingest-cohere">
       Cohere
     </button>
+    <button role="tab"
+            aria-selected="true"
+            aria-controls="infer-api-ingest-hf-tab"
+            id="infer-api-ingest-hf">
+      HuggingFace
+    </button>
     <button role="tab"
             aria-selected="false"
             aria-controls="infer-api-ingest-openai-tab"
@@ -22,6 +28,17 @@
 
 include::infer-api-ingest-pipeline.asciidoc[tag=cohere]
 
+++++
+  </div>
+  <div tabindex="0"
+       role="tabpanel"
+       id="infer-api-ingest-hf-tab"
+       aria-labelledby="infer-api-ingest-hf"
+       hidden="">
+++++
+
+include::infer-api-ingest-pipeline.asciidoc[tag=hugging-face]
+
 ++++
   </div>
   <div tabindex="0"

docs/reference/tab-widgets/inference-api/infer-api-ingest-pipeline.asciidoc

@@ -35,6 +35,31 @@ and the `output_field` that will contain the {infer} results.
 
 // end::cohere[]
 
+// tag::hugging-face[]
+
+[source,console]
+--------------------------------------------------
+PUT _ingest/pipeline/hugging_face_embeddings
+{
+  "processors": [
+    {
+      "inference": {
+        "model_id": "hugging_face_embeddings", <1>
+        "input_output": { <2>
+          "input_field": "content",
+          "output_field": "content_embedding"
+        }
+      }
+    }
+  ]
+}
+--------------------------------------------------
+<1> The name of the inference endpoint you created by using the
+<<put-inference-api>>, it's referred to as `inference_id` in that step.
+<2> Configuration object that defines the `input_field` for the {infer} process
+and the `output_field` that will contain the {infer} results.
+
+// end::hugging-face[]
 
 // tag::openai[]
 

docs/reference/tab-widgets/inference-api/infer-api-mapping-widget.asciidoc

@@ -7,6 +7,12 @@
             id="infer-api-mapping-cohere">
       Cohere
     </button>
+    <button role="tab"
+            aria-selected="false"
+            aria-controls="infer-api-mapping-hf-tab"
+            id="infer-api-mapping-hf">
+      HuggingFace
+    </button>
     <button role="tab"
             aria-selected="false"
             aria-controls="infer-api-mapping-openai-tab"
@@ -22,6 +28,16 @@
 
 include::infer-api-mapping.asciidoc[tag=cohere]
 
+++++
+  </div>
+  <div tabindex="0"
+       role="tabpanel"
+       id="infer-api-mapping-hf-tab"
+       aria-labelledby="infer-api-mapping-hf">
+++++
+
+include::infer-api-mapping.asciidoc[tag=hugging-face]
+
 ++++
   </div>
   <div tabindex="0"

docs/reference/tab-widgets/inference-api/infer-api-mapping.asciidoc

@@ -31,6 +31,37 @@ the {infer} pipeline configuration in the next step.
 
 // end::cohere[]
 
+// tag::hugging-face[]
+
+[source,console]
+--------------------------------------------------
+PUT hugging-face-embeddings
+{
+  "mappings": {
+    "properties": {
+      "content_embedding": { <1>
+        "type": "dense_vector", <2>
+        "dims": 768, <3>
+        "element_type": "float"
+      },
+      "content": { <4>
+        "type": "text" <5>
+      }
+    }
+  }
+}
+--------------------------------------------------
+<1> The name of the field to contain the generated tokens. It must be refrenced
+in the {infer} pipeline configuration in the next step.
+<2> The field to contain the tokens is a `dense_vector` field.
+<3> The output dimensions of the model. Find this value in the
+https://huggingface.co/sentence-transformers/all-mpnet-base-v2[HuggingFace model documentation].
+<4> The name of the field from which to create the dense vector representation.
+In this example, the name of the field is `content`. It must be referenced in
+the {infer} pipeline configuration in the next step.
+<5> The field type which is text in this example.
+
+// end::hugging-face[]
 
 // tag::openai[]
 

docs/reference/tab-widgets/inference-api/infer-api-reindex-widget.asciidoc

@@ -7,6 +7,12 @@
             id="infer-api-reindex-cohere">
       Cohere
     </button>
+    <button role="tab"
+            aria-selected="true"
+            aria-controls="infer-api-reindex-hf-tab"
+            id="infer-api-reindex-hf">
+      HuggingFace
+    </button>
     <button role="tab"
             aria-selected="false"
             aria-controls="infer-api-reindex-openai-tab"
@@ -22,6 +28,17 @@
 
 include::infer-api-reindex.asciidoc[tag=cohere]
 
+++++
+  </div>
+  <div tabindex="0"
+       role="tabpanel"
+       id="infer-api-reindex-hf-tab"
+       aria-labelledby="infer-api-reindex-hf"
+       hidden="">
+++++
+
+include::infer-api-reindex.asciidoc[tag=hugging-face]
+
 ++++
   </div>
   <div tabindex="0"

docs/reference/tab-widgets/inference-api/infer-api-reindex.asciidoc

@@ -25,6 +25,29 @@ may affect the throughput of the reindexing process.
 
 // end::cohere[]
 
+// tag::hugging-face[]
+
+[source,console]
+----
+POST _reindex?wait_for_completion=false
+{
+  "source": {
+    "index": "test-data",
+    "size": 50 <1>
+  },
+  "dest": {
+    "index": "hugging-face-embeddings",
+    "pipeline": "hugging_face_embeddings"
+  }
+}
+----
+// TEST[skip:TBD]
+<1> The default batch size for reindexing is 1000. Reducing `size` to a smaller
+number makes the update of the reindexing process quicker which enables you to
+follow the progress closely and detect errors early.
+
+// end::hugging-face[]
+
 
 // tag::openai[]
 

docs/reference/tab-widgets/inference-api/infer-api-requirements-widget.asciidoc

@@ -7,6 +7,12 @@
             id="infer-api-requirements-cohere">
       Cohere
     </button>
+    <button role="tab"
+            aria-selected="false"
+            aria-controls="infer-api-requirements-hf-tab"
+            id="infer-api-requirements-hf">
+      HuggingFace
+    </button>
     <button role="tab"
             aria-selected="false"
             aria-controls="infer-api-requirements-openai-tab"
@@ -22,6 +28,17 @@
 
 include::infer-api-requirements.asciidoc[tag=cohere]
 
+++++
+  </div>
+  <div tabindex="0"
+       role="tabpanel"
+       id="infer-api-requirements-hf-tab"
+       aria-labelledby="infer-api-requirements-hf"
+       hidden="">
+++++
+
+include::infer-api-requirements.asciidoc[tag=hugging-face]
+
 ++++
   </div>
   <div tabindex="0"

docs/reference/tab-widgets/inference-api/infer-api-requirements.asciidoc

@@ -5,6 +5,12 @@ the Cohere service.
 
 // end::cohere[]
 
+// tag::hugging-face[]
+
+A https://huggingface.co/[HuggingFace account] is required to use the {infer}
+API with the HuggingFace service.
+
+// end::hugging-face[]
 
 // tag::openai[]
 

docs/reference/tab-widgets/inference-api/infer-api-search-widget.asciidoc

@@ -7,6 +7,12 @@
             id="infer-api-search-cohere">
       Cohere
     </button>
+    <button role="tab"
+            aria-selected="true"
+            aria-controls="infer-api-search-hf-tab"
+            id="infer-api-search-hf">
+      HuggingFace
+    </button>
     <button role="tab"
             aria-selected="false"
             aria-controls="infer-api-search-openai-tab"
@@ -22,6 +28,17 @@
 
 include::infer-api-search.asciidoc[tag=cohere]
 
+++++
+  </div>
+  <div tabindex="0"
+       role="tabpanel"
+       id="infer-api-search-hf-tab"
+       aria-labelledby="infer-api-search-hf"
+       hidden="">
+++++
+
+include::infer-api-search.asciidoc[tag=hugging-face]
+
 ++++
   </div>
   <div tabindex="0"

docs/reference/tab-widgets/inference-api/infer-api-search.asciidoc

@@ -72,6 +72,79 @@ query from the `cohere-embeddings` index sorted by their proximity to the query:
 
 // end::cohere[]
 
+// tag::hugging-face[]
+
+[source,console]
+--------------------------------------------------
+GET hugging-face-embeddings/_search
+{
+  "knn": {
+    "field": "content_embedding",
+    "query_vector_builder": {
+      "text_embedding": {
+        "model_id": "hugging_face_embeddings",
+        "model_text": "What's margin of error?"
+      }
+    },
+    "k": 10,
+    "num_candidates": 100
+  },
+  "_source": [
+    "id",
+    "content"
+  ]
+}
+--------------------------------------------------
+// TEST[skip:TBD]
+
+As a result, you receive the top 10 documents that are closest in meaning to the
+query from the `hugging-face-embeddings` index sorted by their proximity to the query:
+
+[source,consol-result]
+--------------------------------------------------
+"hits": [
+      {
+        "_index": "hugging-face-embeddings",
+        "_id": "ljEfo44BiUQvMpPgT20E",
+        "_score": 0.8522128,
+        "_source": {
+          "id": 7960255,
+          "content": "The margin of error can be defined by either of the following equations. Margin of error = Critical value x Standard deviation of the statistic. Margin of error = Critical value x Standard error of the statistic. If you know the standard deviation of the statistic, use the first equation to compute the margin of error. Otherwise, use the second equation. Previously, we described how to compute the standard deviation and standard error."
+        }
+      },
+      {
+        "_index": "hugging-face-embeddings",
+        "_id": "lzEfo44BiUQvMpPgT20E",
+        "_score": 0.7865497,
+        "_source": {
+          "id": 7960259,
+          "content": "1 y ou are told only the size of the sample and are asked to provide the margin of error for percentages which are not (yet) known. 2  This is typically the case when you are computing the margin of error for a survey which is going to be conducted in the future."
+        }
+      },
+      {
+        "_index": "hugging-face-embeddings1",
+        "_id": "DjEfo44BiUQvMpPgT20E",
+        "_score": 0.6229427,
+        "_source": {
+          "id": 2166183,
+          "content": "1. In general, the point at which gains equal losses. 2. In options, the market price that a stock must reach for option buyers to avoid a loss if they exercise. For a call, it is the strike price plus the premium paid. For a put, it is the strike price minus the premium paid."
+        }
+      },
+      {
+        "_index": "hugging-face-embeddings1",
+        "_id": "VzEfo44BiUQvMpPgT20E",
+        "_score": 0.6034223,
+        "_source": {
+          "id": 2173417,
+          "content": "How do you find the area of a circle? Can you measure the area of a circle and use that to find a value for Pi?"
+        }
+      },
+      (...)
+    ]
+--------------------------------------------------
+// NOTCONSOLE
+
+// end::hugging-face[]
 
 // tag::openai[]
 

docs/reference/tab-widgets/inference-api/infer-api-task-widget.asciidoc

@@ -6,6 +6,12 @@
             aria-controls="infer-api-task-cohere-tab"
             id="infer-api-task-cohere">
       Cohere
+    </button>
+        <button role="tab"
+            aria-selected="false"
+            aria-controls="infer-api-task-hf-tab"
+            id="infer-api-task-hf">
+      HuggingFace
     </button>
     <button role="tab"
             aria-selected="false"
@@ -22,6 +28,17 @@
 
 include::infer-api-task.asciidoc[tag=cohere]
 
+++++
+  </div>
+  <div tabindex="0"
+       role="tabpanel"
+       id="infer-api-task-hf-tab"
+       aria-labelledby="infer-api-task-hf"
+       hidden="">
+++++
+
+include::infer-api-task.asciidoc[tag=hugging-face]
+
 ++++
   </div>
   <div tabindex="0"

docs/reference/tab-widgets/inference-api/infer-api-task.asciidoc

@@ -32,6 +32,36 @@ the `cosine` measures are equivalent.
 
 // end::cohere[]
 
+// tag::hugging-face[]
+
+First, you need to create a new {infer} endpoint on
+https://ui.endpoints.huggingface.co/[the Hugging Face endpoint page] to get an
+endpoint URL. Select the model `all-mpnet-base-v2` on the new endpoint creation
+page, then select the `Sentence Embeddings` task under the Advanced
+configuration section. Create the endpoint. Copy the URL after the endpoint
+initialization has been finished, you need the URL in the following {infer} API
+call.
+
+[source,console]
+------------------------------------------------------------
+PUT _inference/text_embedding/hugging_face_embeddings <1>
+{
+  "service": "hugging_face",
+  "service_settings": {
+    "api_key": "<access_token>", <2>
+    "url": "<url_endpoint>" <3>
+  }
+}
+------------------------------------------------------------
+// TEST[skip:TBD]
+<1> The task type is `text_embedding` in the path and the `inference_id` which 
+is the unique identifier of the {infer} endpoint is `hugging_face_embeddings`.
+<2> A valid HuggingFace access token. You can find on the
+https://huggingface.co/settings/tokens[settings page of your account].
+<3> The {infer} endpoint URL you created on Hugging Face.
+
+// end::hugging-face[]
+
 
 // tag::openai[]