Update pro docs

CHANGELOG.md

@@ -2,6 +2,9 @@
 
 ## [Unreleased]
 ### Added
+- (pro) [Autoquality](https://docs.imgproxy.net/autoquality).
+- (pro) [Object detection](https://docs.imgproxy.net/object_detection): `obj` [gravity](https://docs.imgproxy.net/generating_the_url?id=gravity) type, [blur_detections](https://docs.imgproxy.net/generating_the_url?id=blur-detections) processing option, [draw_detections](https://docs.imgproxy.net/generating_the_url?id=draw-detections) processing option.
+- (pro) [Chained pipelines](https://docs.imgproxy.net/chained_pipelines)
 - `IMGPROXY_FALLBACK_IMAGE_HTTP_CODE` config.
 - [expires](https://docs.imgproxy.net/generating_the_url?id=expires) processing option.
 - [skip processing](https://docs.imgproxy.net/generating_the_url?id=skip-processing) processing option.

docs/_sidebar.md

@@ -6,6 +6,7 @@
 * [Signing the URL](signing_the_url)
 * [Watermark](watermark)
 * [Presets](presets)
+* [Chained pipelines <img class='pro-badge' src='assets/pro.svg' alt='pro' />](chained_pipelines)
 * [Serving local files](serving_local_files)
 * [Serving files from Amazon S3](serving_files_from_s3)
 * [Serving files from Google Cloud Storage](serving_files_from_google_cloud_storage)

docs/autoquality.md

@@ -0,0 +1,124 @@
+# Autoquality<img class='pro-badge' src='assets/pro.svg' alt='pro' />
+
+imgproxy can calculate quality for your resulting images so they fit the selected metric the best. The supported methods are [none](#none), [size](#autoquality-by-file-size), [dssim](#autoquality-by-dssim), and [ml](#autoquality-with-ml).
+
+**⚠️Warning:** Autoquality requires the image to be saved several times. Use it only when you prefer the resulting size and quality over the speed.
+
+You can enable autoquality with [config](configuration.md#autoquality) (for all images) or with [processing options](generating_the_url.md#autoquality) (for each image individually).
+
+## None
+
+Disable the autoquality.
+
+**Method name:** `none`
+
+#### Config example
+
+```bash
+IMGPROXY_AUTOQUALITY_METHOD="none"
+```
+
+#### Processing options example
+
+```
+.../autoquality:none/...
+```
+
+## Autoquality by file size
+
+With this method, imgproxy will try to degrade the quality so your image fit the desired file size.
+
+**Method name:** `size`
+**Target:** desired file size
+
+#### Config example
+
+```bash
+IMGPROXY_AUTOQUALITY_METHOD="size"
+# Change value to the desired size in bytes
+IMGPROXY_AUTOQUALITY_TARGET=10240
+IMGPROXY_AUTOQUALITY_MIN=10
+IMGPROXY_AUTOQUALITY_MAX=80
+# Quality 50 for AVIF is pretty the same as 80 for JPEG
+IMGPROXY_AUTOQUALITY_FORMAT_MAX="avif=50"
+```
+
+#### Processing options example
+
+```
+.../autoquality:size:10240:10:80/...
+```
+
+## Autoquality by DSSIM
+
+With this method imgproxy will try to select the quality so the saved image would have the desired [DSSIM](https://en.wikipedia.org/wiki/Structural_similarity#Structural_Dissimilarity) value.
+
+**Method name:** `dssim`
+**Target:** desired DSSIM value
+
+#### Config example
+
+```bash
+IMGPROXY_AUTOQUALITY_METHOD="dssim"
+# Change value to the desired DSSIM
+IMGPROXY_AUTOQUALITY_TARGET=0.02
+# We're happy enough if the resulting DSSIM will differ from the desired by 0.001
+IMGPROXY_AUTOQUALITY_ALLOWED_ERROR=0.001
+IMGPROXY_AUTOQUALITY_MIN=70
+IMGPROXY_AUTOQUALITY_MAX=80
+# Quality 50 for AVIF is pretty the same as 80 for JPEG
+IMGPROXY_AUTOQUALITY_FORMAT_MIN="avif=40"
+IMGPROXY_AUTOQUALITY_FORMAT_MAX="avif=50"
+```
+
+#### Processing options example
+
+```
+.../autoquality:dssim:0.02:70:80:0.001/...
+```
+
+## Autoquality with ML
+
+This method is almost the same as [DSSIM](#autoquality-by-dssim) but imgproxy will try to predict the initial quality using neural networks. Requires neural networs to be configured (see the config examlpe or the config documentation). If a neural network for the resulting format is not provided, [DSSIM](#autoquality-by-dssim) method will be used instead.
+
+**📝Note:** When this method is used, imgproxy will save JPEG images with the most optimal [advanced JPEG compression](configuration.md#advanced-jpeg-compression) settings ignoring config and processing options.
+
+**Method name:** `ml`
+**Target:** desired DSSIM value
+
+#### Config example
+
+```bash
+IMGPROXY_AUTOQUALITY_METHOD="ml"
+# Change value to the desired DSSIM
+IMGPROXY_AUTOQUALITY_TARGET=0.02
+# We're happy enough if the resulting DSSIM will differ from the desired by 0.001
+IMGPROXY_AUTOQUALITY_ALLOWED_ERROR=0.001
+IMGPROXY_AUTOQUALITY_MIN=70
+IMGPROXY_AUTOQUALITY_MAX=80
+# Quality 50 for AVIF is pretty the same as 80 for JPEG
+IMGPROXY_AUTOQUALITY_FORMAT_MIN="avif=40"
+IMGPROXY_AUTOQUALITY_FORMAT_MAX="avif=50"
+# Neural networks paths for JPEG, WebP, and AVIF
+IMGPROXY_AUTOQUALITY_JPEG_NET="/networks/autoquality-jpeg.pb"
+IMGPROXY_AUTOQUALITY_WEBP_NET="/networks/autoquality-webp.pb"
+IMGPROXY_AUTOQUALITY_AVIF_NET="/networks/autoquality-avif.pb"
+```
+
+**📝Note:** If you trust your autoquality neural network, you may want to set `IMGPROXY_AUTOQUALITY_ALLOWED_ERROR` to 1 (maximum possible DSSIM value). In this case, imgproxy will always use the quality predicted by the neural network.
+
+#### Processing options example
+
+```
+.../autoquality:ml:0.02:70:80:0.001/...
+```
+
+### Neural networks format
+
+Neural networs should fit the following requirements:
+* Tensorflow frozen graph format.
+* Input layer size is 416x416.
+* Output layer size is 1x100
+* Output layer values are logits of quality probabilities.
+
+If you're an imgproxy Pro user and you want to train your own network but you don't know how, feel free to contact the imgproxy team for intructions.

docs/chained_pipelines.md

@@ -0,0 +1,58 @@
+# Chained pipelines<img class='pro-badge' src='assets/pro.svg' alt='pro' />
+
+Though imgproxy's [processing pipeline](about_processing_pipeline.md) is suitable for most cases, sometimes it's handy to run multiple chained pipelines with different options.
+
+imgproxy Pro allows you to start a new pipeline by inserting a section with a minus sign (`-`) to the URL path:
+
+```
+.../width:500/crop:1000/-/trim:10/...
+                        ^ the new pipeline starts here
+```
+
+### Example 1: Multiple watermarks
+
+If you need to place multiple watermarks on the same image, you can use chained pipelines for that:
+
+```
+.../rs:fit:500:500/wm:0.5:nowe/wmu:aW1hZ2UxCg/-/wm:0.7:soea/wmu:aW1hZ2UyCg/...
+```
+
+In this example, the first pipeline resizes the image and places the first watermark, and the second pipeline places the second watermark.
+
+### Example 2: Fast trim
+
+The `trim` operation is pretty heavy as it involves loading the whole image to the memory at the very start of processing. However, if you're going to scale down your image and the trim accuracy is not very important to you, it's better to move trimming to a separate pipeline.
+
+```
+.../rs:fit:500:500/-/trim:10/...
+```
+
+In this example, the first pipeline resizes the image, and the second pipeline trims the result. Since the result of the first pipeline is already resized and loaded to the memory, trimming will be done much faster.
+
+## Using with presets
+
+You can use presets in your chained pipelines, and you can use chained pipelines in your presets. However, the behaior may be not obvious. The rules are the following:
+
+* Prest is applied to the pipeline where is was used.
+* Preset may contain chained pipelined, and ones will be chained to the pipeline where the preset was used.
+* Chained pipelines from the preset and from the URL are merged.
+
+### Example
+
+If we have the following preset
+
+```
+test=width:300/height:300/-/width:200/height:200/-/width:100/height:200
+```
+
+and the following URL
+
+```
+.../width:400/-/preset:test/width:500/-/width:600/...
+```
+
+The result will look like this:
+
+```
+.../width:400/-/width:500/height:300/-/width:600/height:200/-/width:100/height:200/...
+```

docs/chaining_the_processing.md

@@ -0,0 +1,58 @@
+# Chaining the processing<img class='pro-badge' src='assets/pro.svg' alt='pro' />
+
+Though imgproxy's [processing pipeline](about_processing_pipeline.md) is suitable for most cases, sometimes it's handy to run multiple chained pipelines with different options.
+
+imgproxy Pro allows you to start a new pipeline by inserting a section with a minus sign (`-`) to the URL path:
+
+```
+.../width:500/crop:1000/-/trim:10/...
+                        ^ the new pipeline starts here
+```
+
+### Example 1: Multiple watermarks
+
+If you need to place multiple watermarks on the same image, you can use chained pipelines for that:
+
+```
+.../rs:fit:500:500/wm:0.5:nowe/wmu:aW1hZ2UxCg/-/wm:0.7:soea/wmu:aW1hZ2UyCg/...
+```
+
+In this example, the first pipeline resizes the image and places the first watermark, and the second pipeline places the second watermark.
+
+### Example 2: Fast trim
+
+The `trim` operation is pretty heavy as it involves loading the whole image to the memory at the very start of processing. However, if you're going to scale down your image and the trim accuracy is not very important to you, it's better to move trimming to a separate pipeline.
+
+```
+.../rs:fit:500:500/-/trim:10/...
+```
+
+In this example, the first pipeline resizes the image, and the second pipeline trims the result. Since the result of the first pipeline is already resized and loaded to the memory, trimming will be done much faster.
+
+## Using with presets
+
+You can use presets in your chained pipelines, and you can use chained pipelines in your presets. However, the behaior may be not obvious. The rules are the following:
+
+* Prest is applied to the pipeline where is was used.
+* Preset may contain chained pipelined, and ones will be chained to the pipeline where the preset was used.
+* Chained pipelines from the preset and from the URL are merged.
+
+### Example
+
+If we have the following preset
+
+```
+test=width:300/height:300/-/width:200/height:200/-/width:100/height:200
+```
+
+and the following URL
+
+```
+.../width:400/-/preset:test/width:500/-/width:600/...
+```
+
+The result will look like this:
+
+```
+.../width:400/-/width:500/height:300/-/width:600/height:200/-/width:100/height:200/...
+```

docs/configuration.md

@@ -125,6 +125,24 @@ Also you may want imgproxy to respond with the same error message that it writes
 
 * `IMGPROXY_AVIF_SPEED`: controls the CPU effort spent improving compression. 0 slowest - 8 fastest. Default: `5`;
 
+### Autoquality
+
+imgproxy can calculate the quality of the resulting image based on selected metric. Read more in the [Autoquality](autoquality.md) guide.
+
+**⚠️Warning:** Autoquality requires the image to be saved several times. Use it only when you prefer the resulting size and quality over the speed.
+
+* `IMGPROXY_AUTOQUALITY_METHOD`: the method of quality calculation. Default: `none`.
+* `IMGPROXY_AUTOQUALITY_TARGET`: desired value of the autoquality method metric. Default: 0.02.
+* `IMGPROXY_AUTOQUALITY_MIN`: minimal quality imgproxy can use. Default: 70.
+* `IMGPROXY_AUTOQUALITY_FORMAT_MIN`: minimal quality imgproxy can use per format, comma divided. Example: `jpeg=70,avif=40,webp=60`. When value for the resulting format is not set, `IMGPROXY_AUTOQUALITY_MIN` value is used. Default: `avif=40`.
+* `IMGPROXY_AUTOQUALITY_MAX`: maximal quality imgproxy can use. Default: 80.
+* `IMGPROXY_AUTOQUALITY_FORMAT_MAX`: maximal quality imgproxy can use per format, comma divided. Example: `jpeg=70,avif=40,webp=60`. When value for the resulting format is not set, `IMGPROXY_AUTOQUALITY_MAX` value is used. Default: `avif=50`.
+* `IMGPROXY_AUTOQUALITY_ALLOWED_ERROR`: allowed `IMGPROXY_AUTOQUALITY_TARGET` error. Applicable only to `dssim` and `ml` methods. Default: 0.001.
+* `IMGPROXY_AUTOQUALITY_MAX_RESOLUTION`: when value is greater then zero and the result resolution exceeds the value, autoquality won't be used. Default: 0.
+* `IMGPROXY_AUTOQUALITY_JPEG_NET`: path to the neural network for JPEG.
+* `IMGPROXY_AUTOQUALITY_WEBP_NET`: path to the neural network for WebP.
+* `IMGPROXY_AUTOQUALITY_AVIF_NET`: path to the neural network for AVIF.
+
 ## AVIF/WebP support detection
 
 imgproxy can use the `Accept` HTTP header to detect if the browser supports AVIF or WebP and use it as the default format. This feature is disabled by default and can be enabled by the following options:
@@ -182,6 +200,17 @@ imgproxy Pro can apply unsharpening mask to your images.
 * `IMGPROXY_UNSHARPENING_WEIGHT`: <img class='pro-badge' src='assets/pro.svg' alt='pro' /> a floating-point number that defines how neighbor pixels will affect the current pixel. Greater the value - sharper the image. Should be greater than zero. Default: `1`.
 * `IMGPROXY_UNSHARPENING_DIVIDOR`: <img class='pro-badge' src='assets/pro.svg' alt='pro' /> a floating-point number that defines the unsharpening strength. Lesser the value - sharper the image. Should be greater than zero. Default: `24`.
 
+## Object detection
+
+imgproxy can detect objects on the image and use them for smart crop, bluring the detections, or drawing the detections.
+
+* `IMGPROXY_OBJECT_DETECTION_CONFIG`: path to the neural network config. Default: blank.
+* `IMGPROXY_OBJECT_DETECTION_WEIGHTS`: path to the neural network weights. Default: blank.
+* `IMGPROXY_OBJECT_DETECTION_CLASSES`: path to the text file with the classes names, one by line. Default: blank.
+* `IMGPROXY_OBJECT_DETECTION_NET_SIZE`: the size of the neural network input. The width and the heights of the inputs should be the same, so this config value should be a single number. Default: 416.
+* `IMGPROXY_OBJECT_DETECTION_CONFIDENCE_THRESHOLD`: the detections with confidences below this value will be discarded. Default: 0.2.
+* `IMGPROXY_OBJECT_DETECTION_NMS_THRESHOLD`: non max supression threshold. Don't change this if you don't know what you're doing. Default: 0.4.
+
 ## Fallback image
 
 You can set up a fallback image that will be used in case imgproxy can't fetch the requested one. Use one of the following variables:

docs/generating_the_url.md

@@ -180,6 +180,7 @@ Default: `ce:0:0`
 **Special gravities**:
 
 * `gravity:sm` - smart gravity. `libvips` detects the most "interesting" section of the image and considers it as the center of the resulting image. Offsets are not applicable here;
+* `gravity:obj:%class_name1:%class_name2:...:%class_nameN` - <img class='pro-badge' src='assets/pro.svg' alt='pro' /> object-oriented gravity. imgproxy [detects objects](object_detection.md) of provided classes on the image and calculates the resulting image center using their positions. If class names are omited, imgproxy will use all the detected objects.
 * `gravity:fp:%x:%y` - focus point gravity. `x` and `y` are floating point numbers between 0 and 1 that define the coordinates of the center of the resulting image. Treat 0 and 1 as right/left for `x` and top/bottom for `y`.
 
 ### Crop
@@ -372,6 +373,26 @@ ush:%mode:%weight:%dividor
 
 Allows redefining unsharpening options. All arguments have the same meaning as [Unsharpening](configuration.md#unsharpening) configs. All arguments are optional and can be omitted.
 
+### Blur detections<img class='pro-badge' src='assets/pro.svg' alt='pro' /> :id=blur-detections
+
+```
+blur_detections:%sigma:%class_name1:%class_name2:...:%class_nameN
+bd:%sigma:%class_name1:%class_name2:...:%class_nameN
+```
+
+imgproxy [detects objects](object_detection.md) of provided classes and blus them. If class names are omitted, imgproxy blurs all the detected objects.
+
+`sigma` defines the size of a mask imgproxy will use.
+
+### Draw detections<img class='pro-badge' src='assets/pro.svg' alt='pro' /> :id=draw-detections
+
+```
+draw_detections:%draw:%class_name1:%class_name2:...:%class_nameN
+dd:%draw:%class_name1:%class_name2:...:%class_nameN
+```
+
+When `draw` is set to `1`, `t` or `true`, imgproxy [detects objects](object_detection.md) of provided classes and draws their bounding boxed. If class names are omitted, imgproxy draws the bounding boxes of all the detected objects.
+
 ### Watermark
 
 ```
@@ -398,7 +419,7 @@ Puts watermark on the processed image.
 
 Default: disabled
 
-### Watermark URL<img class='pro-badge' src='assets/pro.svg' alt='pro' /> :id=watermark
+### Watermark URL<img class='pro-badge' src='assets/pro.svg' alt='pro' /> :id=watermark-url
 
 ```
 watermark_url:%url
@@ -449,6 +470,17 @@ Redefines quality of the resulting image, percentage. When `0`, quality is assum
 
 Default: 0.
 
+### Autoquality
+
+```
+autoquality:%method:%target:%min_quality:%max_quality:%allowed_error
+aq:%method:%target:%min_quality:%max_quality:%allowed_error
+```
+
+Redefines autoqiality settings. All arguments have the same meaning as [Autoquality](configuration.md#autoqiality) configs. All arguments are optional and can be omitted.
+
+**⚠️Warning:** Autoquality requires the image to be saved several times. Use it only when you prefer the resulting size and quality over the speed.
+
 ### Max Bytes
 
 ```

docs/getting_the_image_info.md

@@ -47,7 +47,10 @@ imgproxy responses with JSON body and returns the following info:
 * `width`: image/video width;
 * `height`: image/video height;
 * `size`: file size. Can be zero if the image source doesn't set `Content-Length` header properly;
-* `exif`: JPEG exif data.
+* `exif`: Exif data;
+* `iptc`: IPTC data.
+
+**📝Note:** There are lots of IPTC tags in the spec, but imgproxy supports only a few of them. If you need some tags to be supported, just contact us.
 
 #### Example (JPEG)
 
@@ -63,6 +66,12 @@ imgproxy responses with JSON body and returns the following info:
     "Date and Time": "2016:09:11 22:15:03",
     "Model": "NIKON D810",
     "Software": "Adobe Photoshop Lightroom 6.1 (Windows)"
+  },
+  "iptc": {
+    "Name": "Spider-Man",
+    "Caption": "Spider-Man swings on the web",
+    "Copyright Notice": "Daily Bugle",
+    "Keywords": ["spider-man", "menance", "offender"]
   }
 }
 ```

docs/object_detection.md

@@ -0,0 +1,41 @@
+# Object detection<img class='pro-badge' src='assets/pro.svg' alt='pro' />
+
+imgproxy can detect objects on the image and use them for smart crop, bluring the detections, or drawing the detections.
+
+For object detection purposes, imgproxy uses [Darknet YOLO](https://github.com/AlexeyAB/darknet) model. We provide Docker images with a model trained for face detection, but you can use any Darknet YOLO model found in the [zoo](https://github.com/AlexeyAB/darknet/wiki/YOLOv4-model-zoo) or you can train your own model following the [guide](https://github.com/AlexeyAB/darknet#how-to-train-to-detect-your-custom-objects).
+
+## Configuration
+
+You need to define four config variables to enable object detection:
+
+* `IMGPROXY_OBJECT_DETECTION_CONFIG`: path to the neural network config.
+* `IMGPROXY_OBJECT_DETECTION_WEIGHTS`: path to the neural network weights.
+* `IMGPROXY_OBJECT_DETECTION_CLASSES`: path to the text file with the classes names, one by line.
+* `IMGPROXY_OBJECT_DETECTION_NET_SIZE`: the size of the neural network input. The width and the heights of the inputs should be the same, so this config value should be a single number. Default: 416.
+
+Read the [configuration](configuration.md#object-detection) guide for more config values info.
+
+## Usage examples
+### Object-oriented crop
+
+You can [crop](https://docs.imgproxy.net/generating_the_url?id=crop) your images and keep objects of desired classes in frame:
+
+```
+.../crop:256:256/g:obj:face/...
+```
+
+### Bluring detections
+
+You can [blur objects](https://docs.imgproxy.net/generating_the_url?id=blur-detections) of desired classes for anonymization or hiding NSFW content:
+
+```
+.../blur_detections:7:face/...
+```
+
+### Draw detections
+
+You can make imgproxy [draw bounding boxes](https://docs.imgproxy.net/generating_the_url?id=draw-detections) of detected objects of desired classes (handy for testing your models):
+
+```
+.../draw_detections:1:face/...
+```