Explorar o código

Add jemalloc and TCMalloc to Docker image

DarthSim %!s(int64=2) %!d(string=hai) anos
pai
achega
5f6cb64864
Modificáronse 5 ficheiros con 54 adicións e 2 borrados
  1. 1 0
      CHANGELOG.md
  2. 8 0
      docker/Dockerfile
  3. 22 0
      docker/entrypoint.sh
  4. 4 0
      docs/configuration.md
  5. 19 2
      docs/memory_usage_tweaks.md

+ 1 - 0
CHANGELOG.md

@@ -5,6 +5,7 @@
 - Add `process_resident_memory_bytes`, `process_virtual_memory_bytes`, `go_memstats_sys_bytes`, `go_memstats_heap_idle_bytes`, `go_memstats_heap_inuse_bytes`, `go_goroutines`, `go_threads`, `buffer_default_size_bytes`, `buffer_max_size_bytes`, and `buffer_size_bytes` metrics to OpenTelemetry.
 - Add `process_resident_memory_bytes`, `process_virtual_memory_bytes`, `go_memstats_sys_bytes`, `go_memstats_heap_idle_bytes`, `go_memstats_heap_inuse_bytes`, `go_goroutines`, `go_threads`, `buffer_default_size_bytes`, `buffer_max_size_bytes`, and `buffer_size_bytes` metrics to OpenTelemetry.
 - Add support for the `Last-Modified` response header and the `If-Modified-Since` request header (controlled by the `IMGPROXY_USE_LAST_MODIFIED` config).
 - Add support for the `Last-Modified` response header and the `If-Modified-Since` request header (controlled by the `IMGPROXY_USE_LAST_MODIFIED` config).
 - Add `IMGPROXY_S3_ASSUME_ROLE_ARN` config.
 - Add `IMGPROXY_S3_ASSUME_ROLE_ARN` config.
+- Add `IMGPROXY_MALLOC` Docker-only config.
 
 
 ### Change
 ### Change
 - Optimized memory buffers pooling for better performance and memory reusage.
 - Optimized memory buffers pooling for better performance and memory reusage.

+ 8 - 0
docker/Dockerfile

@@ -25,16 +25,23 @@ RUN apt-get update \
     libpcre3 \
     libpcre3 \
     fontconfig-config \
     fontconfig-config \
     media-types \
     media-types \
+    libjemalloc2 \
+    libtcmalloc-minimal4 \
+  && ln -s /usr/lib/$(uname -m)-linux-gnu/libjemalloc.so.2 /usr/local/lib/libjemalloc.so \
+  && ln -s /usr/lib/$(uname -m)-linux-gnu/libtcmalloc_minimal.so.4 /usr/local/lib/libtcmalloc_minimal.so \
   && rm -rf /var/lib/apt/lists/*
   && rm -rf /var/lib/apt/lists/*
 
 
 COPY --from=0 /usr/local/bin/imgproxy /usr/local/bin/
 COPY --from=0 /usr/local/bin/imgproxy /usr/local/bin/
 COPY --from=0 /usr/local/lib /usr/local/lib
 COPY --from=0 /usr/local/lib /usr/local/lib
 
 
+COPY docker/entrypoint.sh /usr/local/bin/
+
 COPY NOTICE /usr/local/share/doc/imgproxy/
 COPY NOTICE /usr/local/share/doc/imgproxy/
 
 
 ENV VIPS_WARNING=0
 ENV VIPS_WARNING=0
 ENV MALLOC_ARENA_MAX=2
 ENV MALLOC_ARENA_MAX=2
 ENV LD_LIBRARY_PATH /usr/local/lib
 ENV LD_LIBRARY_PATH /usr/local/lib
+ENV IMGPROXY_MALLOC malloc
 
 
 RUN groupadd -r imgproxy \
 RUN groupadd -r imgproxy \
   && useradd -r -u 999 -g imgproxy imgproxy \
   && useradd -r -u 999 -g imgproxy imgproxy \
@@ -42,6 +49,7 @@ RUN groupadd -r imgproxy \
   && chmod 777 /var/cache/fontconfig
   && chmod 777 /var/cache/fontconfig
 USER 999
 USER 999
 
 
+ENTRYPOINT [ "entrypoint.sh" ]
 CMD ["imgproxy"]
 CMD ["imgproxy"]
 
 
 EXPOSE 8080
 EXPOSE 8080

+ 22 - 0
docker/entrypoint.sh

@@ -0,0 +1,22 @@
+#!/bin/bash
+
+case "$IMGPROXY_MALLOC" in
+
+  malloc)
+    # Do nothing
+    ;;
+
+  jemalloc)
+    export LD_PRELOAD="$LD_PRELOAD:/usr/local/lib/libjemalloc.so"
+    ;;
+
+  tcmalloc)
+    export LD_PRELOAD="$LD_PRELOAD:/usr/local/lib/libtcmalloc_minimal.so"
+    ;;
+
+  *)
+    echo "Unknows malloc: $IMGPROXY_MALLOC"
+    exit 1
+esac
+
+exec "$@"

+ 4 - 0
docs/configuration.md

@@ -519,6 +519,10 @@ imgproxy can send logs to syslog, but this feature is disabled by default. To en
 * `IMGPROXY_DOWNLOAD_BUFFER_SIZE`: the initial size (in bytes) of a single download buffer. When set to zero, initializes empty download buffers. Default: `0`
 * `IMGPROXY_DOWNLOAD_BUFFER_SIZE`: the initial size (in bytes) of a single download buffer. When set to zero, initializes empty download buffers. Default: `0`
 * `IMGPROXY_FREE_MEMORY_INTERVAL`: the interval (in seconds) at which unused memory will be returned to the OS. Default: `10`
 * `IMGPROXY_FREE_MEMORY_INTERVAL`: the interval (in seconds) at which unused memory will be returned to the OS. Default: `10`
 * `IMGPROXY_BUFFER_POOL_CALIBRATION_THRESHOLD`: the number of buffers that should be returned to a pool before calibration. Default: `1024`
 * `IMGPROXY_BUFFER_POOL_CALIBRATION_THRESHOLD`: the number of buffers that should be returned to a pool before calibration. Default: `1024`
+* `IMGPROXY_MALLOC`: _(Docker only)_ malloc implementation to use. The following implementations are supported:
+  * `malloc`: standard malloc implementation
+  * `jemalloc`: https://jemalloc.net/
+  * `tcmalloc`: https://github.com/google/tcmalloc
 
 
 ## Miscellaneous
 ## Miscellaneous
 
 

+ 19 - 2
docs/memory_usage_tweaks.md

@@ -27,9 +27,11 @@ MALLOC_ARENA_MAX=2 imgproxy
 This will reduce GLib memory appetites by reducing the number of malloc arenas that it can create. By default GLib creates one are per thread, and this would follow to a memory fragmentation.
 This will reduce GLib memory appetites by reducing the number of malloc arenas that it can create. By default GLib creates one are per thread, and this would follow to a memory fragmentation.
 
 
 
 
-### Using jemalloc
+### Using alternative malloc implementations
 
 
-If setting `MALLOC_ARENA_MAX` doesn't show you satisfying results, it's time to try [jemalloc](http://jemalloc.net/). As [jemalloc site](http://jemalloc.net/) says:
+If setting `MALLOC_ARENA_MAX` doesn't show you satisfying results, it's time to try alternative malloc implementations.
+
+#### jemalloc
 
 
 > jemalloc is a general purpose malloc(3) implementation that emphasizes fragmentation avoidance and scalable concurrency support.
 > jemalloc is a general purpose malloc(3) implementation that emphasizes fragmentation avoidance and scalable concurrency support.
 
 
@@ -39,3 +41,18 @@ Most Linux distributives provide their jemalloc packages. Using jemalloc doesn't
 sudo apt-get install libjemalloc2
 sudo apt-get install libjemalloc2
 LD_PRELOAD='/usr/lib/x86_64-linux-gnu/libjemalloc.so.2' imgproxy
 LD_PRELOAD='/usr/lib/x86_64-linux-gnu/libjemalloc.so.2' imgproxy
 ```
 ```
+
+Official imgproxy Docker images starting `v3.17.0` have jemalloc preinstalled. Its usage can be enabled by setting the `IMGPROXY_MALLOC` environment variable to `jemalloc`.
+
+#### TCMalloc
+
+> TCMalloc is Google's customized implementation of C's malloc() and C++'s operator new used for memory allocation within our C and C++ code. TCMalloc is a fast, multi-threaded malloc implementation.
+
+Most Linux distributives provide their TCMalloc packages. Using TCMalloc doesn't require rebuilding imgproxy or it's dependencies and can be enabled by the `LD_PRELOAD` environment variable. See the example with Debian below. Note that TCMalloc library path may vary on your system.
+
+```
+sudo apt-get install libtcmalloc-minimal4
+LD_PRELOAD='/usr/lib/x86_64-linux-gnu/libtcmalloc_minimal.so.4' imgproxy
+```
+
+Official imgproxy Docker images starting `v3.17.0` have TCMalloc preinstalled. Its usage can be enabled by setting the `IMGPROXY_MALLOC` environment variable to `tcmalloc`.