|
@@ -453,6 +453,33 @@ bucket, in this example, named "foo".
|
|
|
The bucket needs to exist to register a repository for snapshots. If you did not
|
|
|
create the bucket then the repository registration will fail.
|
|
|
|
|
|
+===== Cleaning up multi-part uploads
|
|
|
+
|
|
|
+{es} uses S3's multi-part upload process to upload larger blobs to the
|
|
|
+repository. The multi-part upload process works by dividing each blob into
|
|
|
+smaller parts, uploading each part independently, and then completing the
|
|
|
+upload in a separate step. This reduces the amount of data that {es} must
|
|
|
+re-send if an upload fails: {es} only needs to re-send the part that failed
|
|
|
+rather than starting from the beginning of the whole blob. The storage for each
|
|
|
+part is charged independently starting from the time at which the part was
|
|
|
+uploaded.
|
|
|
+
|
|
|
+If a multi-part upload cannot be completed then it must be aborted in order to
|
|
|
+delete any parts that were successfully uploaded, preventing further storage
|
|
|
+charges from accumulating. {es} will automatically abort a multi-part upload on
|
|
|
+failure, but sometimes the abort request itself fails. For example, if the
|
|
|
+repository becomes inaccessible or the instance on which {es} is running is
|
|
|
+terminated abruptly then {es} cannot complete or abort any ongoing uploads.
|
|
|
+
|
|
|
+You must make sure that failed uploads are eventually aborted to avoid
|
|
|
+unnecessary storage costs. You can use the
|
|
|
+https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListMultipartUploads.html[List
|
|
|
+multipart uploads API] to list the ongoing uploads and look for any which are
|
|
|
+unusually long-running, or you can
|
|
|
+https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpu-abort-incomplete-mpu-lifecycle-config.html[configure
|
|
|
+a bucket lifecycle policy] to automatically abort incomplete uploads once they
|
|
|
+reach a certain age.
|
|
|
+
|
|
|
[[repository-s3-aws-vpc]]
|
|
|
[discrete]
|
|
|
==== AWS VPC Bandwidth Settings
|