Version 4.0-rc docs

docs/cache/external.mdx

@@ -0,0 +1,217 @@
+---
+description: External cache options for imgproxy, including CDNs, nginx, and Varnish
+---
+
+# External cache
+
+External cache refers to caching layers in front of imgproxy, such as CDNs, reverse proxies, or caching appliances. These solutions provide edge caching, geo-distribution, and high-performance image delivery.
+
+## CDN options
+
+Most major CDNs (Cloudflare, Fastly, AWS CloudFront, Akamai, etc.) can cache imgproxy responses:
+
+### Recommended CDN settings
+
+* **Image optimization** - Disable CDN image optimization to avoid double-processing
+* **Compression** - In case your CDN supports conditional compression of SVG images, enable gzip/Brotli compression for bandwidth savings
+* **Include essential headers*** - Include `Accept` header in the cache key if you're using [format negotiation](../configuration/options.mdx#avifwebpjpeg-xl-support-detection). If you have client hints enabled, include also `DPR`, `Sec-CH-Dpr`, and `Sec-CH-Width` headers.
+* **Origin shield** - Add a secondary cache layer between your CDN and imgproxy origin if your CDN supports it.
+
+## nginx caching
+
+nginx can act as a reverse proxy cache in front of imgproxy:
+
+```nginx
+proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=imgproxy_cache:10m max_size=50g inactive=30d;
+
+upstream imgproxy {
+  server localhost:8080;
+}
+
+server {
+  listen 80;
+  server_name images.example.com;
+
+  # Uncomment if you do not use format negotiation and client hints
+  # proxy_cache_key "$scheme$proxy_host$request_uri";
+
+  # Uncomment if you use format negotiation
+  # proxy_cache_key "$scheme$proxy_host$request_uri$http_accept";
+
+  # Uncomment if you use client hints
+  # proxy_cache_key "$scheme$request_method$host$request_uri:dpr=$http_dpr:width=$http_width:chdpr=$http_sec_ch_dpr:chwidth=$http_sec_ch_width";
+
+  # Uncomment if you use format negotiation and client hints
+  # proxy_cache_key "$scheme$request_method$host$request_uri:accept=$http_accept:dpr=$http_dpr:width=$http_width:chdpr=$http_sec_ch_dpr:chwidth=$http_sec_ch_width";
+
+  location / {
+    proxy_pass http://imgproxy;
+
+    # Cache configuration
+    proxy_cache imgproxy_cache;
+    proxy_cache_valid 200 30d;
+    proxy_cache_valid 404 1m;
+    proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
+    proxy_cache_background_update on;
+    proxy_cache_lock on;
+    proxy_cache_bypass $http_pragma $http_authorization;
+    proxy_no_cache $http_pragma $http_authorization;
+
+    # Add cache status header for debugging
+    add_header X-Cache-Status $upstream_cache_status;
+  }
+}
+```
+
+### Possible cache key setup
+
+The exact key depends on which imgproxy features can change output for the same URL.
+
+1. Basic key when output depends only on URL/path options:
+
+```nginx
+proxy_cache_key "$scheme$proxy_host$request_uri";
+```
+
+2. Include [format negotiation](../configuration/options.mdx#avifwebpjpeg-xl-support-detection) signal when WebP/AVIF/JXL preference can vary by client:
+
+```nginx
+proxy_cache_key "$scheme$proxy_host$request_uri$http_accept";
+```
+
+3. Include [Client Hints](../configuration/options.mdx#client-hints-support) dimensions when width/DPR affects output:
+
+```nginx
+proxy_cache_key "$scheme$request_method$host$request_uri:accept=$http_accept:dpr=$http_dpr:width=$http_width:chdpr=$http_sec_ch_dpr:chwidth=$http_sec_ch_width";
+```
+
+Use the smallest key that still prevents collisions for your traffic profile.
+
+### Key recommendations
+
+* Include `Accept` in the cache key if [format negotiation](../configuration/options.mdx#avifwebpjpeg-xl-support-detection) is used.
+* Include DPR/width-related hints in the cache key if [Client Hints](../configuration/options.mdx#client-hints-support) are used to vary output.
+* Set long `inactive` timeout (30d) to keep popular images in cache.
+* Configure `proxy_cache_use_stale` for graceful degradation during origin issues.
+* Monitor `/proc/sys/fs/file-max` and increase if needed for large caches.
+* Use separate cache zones for different image types if needed.
+
+## Varnish caching
+
+Varnish is a powerful reverse proxy cache optimized for HTTP performance:
+
+```vcl
+vcl 4.1;
+
+backend imgproxy {
+  .host = "localhost";
+  .port = "8080";
+  .connect_timeout = 600ms;
+  .first_byte_timeout = 600s;
+  .between_bytes_timeout = 60s;
+}
+
+sub vcl_recv {
+  set req.backend_hint = imgproxy;
+
+  # Cache only GET and HEAD requests
+  if (req.method != "GET" && req.method != "HEAD") {
+    return (pass);
+  }
+
+  # Include important headers in cache key
+  if (req.http.Accept) {
+    set req.http.X-Accept = req.http.Accept;
+  }
+
+  # Keep DPR if Client Hints are not enabled
+  if (req.http.DPR) {
+    set req.http.X-DPR = req.http.DPR;
+  }
+
+  if (req.http.Width) {
+    set req.http.X-Width = req.http.Width;
+  }
+
+  if (req.http.Sec-CH-DPR) {
+    set req.http.X-CH-DPR = req.http.Sec-CH-DPR;
+  }
+
+  if (req.http.Sec-CH-Width) {
+    set req.http.X-CH-Width = req.http.Sec-CH-Width;
+  }
+}
+
+sub vcl_hash {
+  # Base hash
+  hash_data(req.url);
+  hash_data(req.http.Host);
+
+  # Include Accept header for format negotiation
+  if (req.http.X-Accept) {
+    hash_data(req.http.X-Accept);
+  }
+
+  # Include DPR if present
+  if (req.http.X-DPR) {
+    hash_data(req.http.X-DPR);
+  }
+
+  if (req.http.X-Width) {
+    hash_data(req.http.X-Width);
+  }
+
+  if (req.http.X-CH-DPR) {
+    hash_data(req.http.X-CH-DPR);
+  }
+
+  if (req.http.X-CH-Width) {
+    hash_data(req.http.X-CH-Width);
+  }
+}
+
+sub vcl_backend_response {
+  # Cache successful responses for 30 days
+  if (beresp.status == 200) {
+    set beresp.ttl = 30d;
+    set beresp.keep = 30d;
+  }
+
+  # Cache 404s for a short period
+  if (beresp.status == 404) {
+    set beresp.ttl = 1m;
+  }
+
+  # Enable cache using stale object if backend is down
+  set beresp.grace = 24h;
+}
+
+sub vcl_deliver {
+  # Add cache hit/miss header
+  if (obj.hits > 0) {
+    set resp.http.X-Cache = "HIT";
+    set resp.http.X-Cache-Hits = obj.hits;
+  } else {
+    set resp.http.X-Cache = "MISS";
+  }
+}
+```
+
+### Key recommendations
+
+* Use `sub vcl_hash` to customize cache key behavior.
+* Include `Accept` for [format negotiation](../configuration/options.mdx#avifwebpjpeg-xl-support-detection).
+* Include DPR/width-related headers when [Client Hints](../configuration/options.mdx#client-hints-support) support is enabled (`DPR`, `Width`, `Sec-CH-DPR`, `Sec-CH-Width`).
+* Set grace period (`beresp.grace`) for graceful error handling.
+* Monitor Varnish statistics with `varnishstat`.
+* Use tags for efficient cache purging: add `set beresp.http.Surrogate-Key = "images:product:123"` in `vcl_backend_response`.
+* Configure workspace limits if caching many large images.
+
+## Cache headers from imgproxy
+
+imgproxy sends the following headers useful for external caching:
+
+* `Cache-Control: public, max-age=31536000` - Long-term caching (1 year default, configurable with `IMGPROXY_TTL`)
+* `ETag` - If `IMGPROXY_USE_ETAG=true` (by default), enables conditional requests
+* `Last-Modified` - If `IMGPROXY_USE_LAST_MODIFIED=true` (by default), enables conditional requests
+* `Content-Type` - Important for cache key if not in `Accept` header

docs/cache/internal.mdx

@@ -0,0 +1,68 @@
+---
+title: Internal cache
+description: Learn about how to configure and use imgproxy's internal cache
+---
+
+# Internal cache ((pro))
+
+imgproxy Pro provides an internal cache that stores processed images on disk or in cloud storage. This cache can act as both a primary and a secondary cache, serving as a fallback when your CDN cache misses.
+
+:::tip
+While putting a CDN in front of imgproxy is and will always be a best practice, the internal cache provides long-term storage for cases that require an additional caching layer.
+:::
+
+## Why use internal cache?
+
+The internal cache provides long-term persistent storage for processed images, unlike CDNs, which typically delete rarely accessed content. It stores images in a single location rather than across multiple edge stores, eliminating cache misses when requests hit different edges. The cache is designed specifically for imgproxy, working seamlessly with features like modern image format detection and client hints support that generic external caches don't understand by default.
+
+The cache is protected by the same security measures as imgproxy itself, including URL signatures and processing restrictions. Importantly, URL signatures are not part of the cache key, so you can rotate keys or use multiple key/salt pairs without invalidating cached images. You maintain full control over where the cache is stored and how it integrates with your infrastructure.
+
+## Configuration
+
+You need to define the following config variables to enable the internal cache:
+
+* [`IMGPROXY_CACHE_USE`]: the cache storage adapter to use. Can be `fs`, `s3`, `gcs`, `abs` (Azure Blob Storage), or `swift` (OpenStack Swift). When blank, the cache is disabled. Default: blank
+* [`IMGPROXY_CACHE_PATH_PREFIX`]: _(optional)_ a path prefix for the cache files. This can be useful to organize cache files in a specific directory structure. Default: blank
+* [`IMGPROXY_CACHE_BUCKET`]: _(optional)_ the bucket name for cloud storage adapters (S3, GCS, ABS, Swift). When using the filesystem adapter, this can be used as an additional path component. Default: blank
+* [`IMGPROXY_CACHE_KEY_HEADERS`]: _(optional)_ a list of HTTP request headers (comma-separated) to include in the cache key. This allows caching different versions of the same image based on request headers. Default: blank
+* [`IMGPROXY_CACHE_KEY_COOKIES`]: _(optional)_ a list of HTTP request cookies (comma-separated) to include in the cache key. This allows caching different versions of the same image based on cookies. Default: blank
+* [`IMGPROXY_CACHE_REPORT_ERRORS`]: When `true`, imgproxy will report cache errors instead of silently falling back to processing without cache. Default: `false`
+
+### Storage configuration
+
+The internal cache supports all the storage backends that imgproxy can read source images from: local filesystem, Amazon S3 and compatible services (Cloudflare R2, DigitalOcean Spaces, MinIO, etc.), Google Cloud Storage, Microsoft Azure Blob Storage, and OpenStack Swift.
+
+Configure the storage backend using `IMGPROXY_CACHE_*` variables:
+
+* For filesystem cache, see [Cache storage: Local filesystem](./internal/local_filesystem.mdx).
+* For S3 cache, see [Cache storage: Amazon S3](./internal/amazon_s3.mdx).
+* For GCS cache, see [Cache storage: Google Cloud Storage](./internal/google_cloud_storage.mdx).
+* For Azure Blob Storage cache, see [Cache storage: Azure Blob Storage](./internal/azure_blob_storage.mdx).
+* For Swift cache, see [Cache storage: OpenStack Object Storage ("Swift")](./internal/openstack_swift.mdx).
+
+## Cache key
+
+The cache key is generated based on:
+
+* Source image URL
+* Processing options
+* Output format
+* Optional: Request headers specified in `IMGPROXY_CACHE_KEY_HEADERS`
+* Optional: Request cookies specified in `IMGPROXY_CACHE_KEY_COOKIES`
+
+URL signature is **not** part of the cache key, allowing key rotation without invalidating the cache.
+
+## Limitations
+
+* **No manual cache invalidation**: Currently, imgproxy doesn't provide a built-in means to invalidate the cache. However, imgproxy includes the [cachebuster](../usage/processing.mdx#cache-buster) in the cache key, so you can use it to force cache invalidation when needed. Most storage offerings also support object expiration, so you can set a reasonable expiration time for cached images.
+* **No cache for info requests**: The internal cache is currently used only for image processing requests. Requests to the `/info` endpoint are not cached.
+
+## How it works
+
+When a request comes in:
+
+1. imgproxy checks the URL signature (if enabled).
+2. imgproxy generates the cache key from the request parameters.
+3. imgproxy checks if a cached image exists in the configured storage.
+4. If the cached image exists and is valid, imgproxy serves it directly.
+5. If not, imgproxy processes the image and stores the result in the cache before serving it.

docs/cache/internal/amazon_s3.mdx

@@ -0,0 +1,72 @@
+---
+description: Learn about how to configure Amazon S3 cache in imgproxy
+---
+
+# Internal cache: Amazon S3
+
+imgproxy can store cached images in Amazon S3 buckets or S3-compatible storage. To use S3 cache, do the following:
+
+1. Set the `IMGPROXY_CACHE_USE` environment variable to `s3`.
+2. [Set up the necessary credentials](#set-up-credentials) to grant access to your cache bucket.
+3. Specify the cache bucket name with `IMGPROXY_CACHE_BUCKET`.
+4. _(optional)_ Specify the AWS region with `IMGPROXY_CACHE_S3_REGION` or `AWS_REGION`. Default: `us-west-1`
+5. _(optional)_ Specify the S3 endpoint with `IMGPROXY_CACHE_S3_ENDPOINT`. You can also set `IMGPROXY_CACHE_S3_ENDPOINT_USE_PATH_STYLE=false` to use the virtual host style for the endpoint.
+6. _(optional)_ Specify the AWS IAM Role to Assume with `IMGPROXY_CACHE_S3_ASSUME_ROLE_ARN`.
+7. _(optional)_ Specify the External ID to pass to the AWS IAM Role when assuming it using `IMGPROXY_CACHE_S3_ASSUME_ROLE_EXTERNAL_ID`. This will have no effect if the assume role ARN is not specified.
+
+### Configuration
+
+* `IMGPROXY_CACHE_USE`: set to `s3` to enable S3 cache.
+* `IMGPROXY_CACHE_BUCKET`: the S3 bucket name for cache storage. Default: blank
+* `IMGPROXY_CACHE_S3_REGION`: the S3 region for the cache bucket. Default: blank
+* `IMGPROXY_CACHE_S3_ENDPOINT`: a custom S3 endpoint for the cache. Useful for S3-compatible services like MinIO, Cloudflare R2, DigitalOcean Spaces, etc. Default: blank
+* `IMGPROXY_CACHE_S3_ENDPOINT_USE_PATH_STYLE`: controls how the S3 bucket endpoint is constructed. When `true`, the endpoint will be constructed using the path style (`https://your-endpoint.com/%bucket`). When `false`, the endpoint will be constructed using the virtual host style (`https://%bucket.your-endpoint.com`). Default: `true`
+* `IMGPROXY_CACHE_S3_ASSUME_ROLE_ARN`: the ARN of an IAM role to assume for cache access. Default: blank
+* `IMGPROXY_CACHE_S3_ASSUME_ROLE_EXTERNAL_ID`: the external ID required to assume the IAM role for cache access. Default: blank
+* `IMGPROXY_CACHE_PATH_PREFIX`: a path prefix for the cache files. Default: blank
+* `IMGPROXY_CACHE_KEY_HEADERS`: a comma-separated list of HTTP request headers to include in the cache key. Default: blank
+* `IMGPROXY_CACHE_KEY_COOKIES`: a comma-separated list of HTTP request cookies to include in the cache key. Default: blank
+* `IMGPROXY_CACHE_REPORT_ERRORS`: when `true`, imgproxy will report cache errors instead of silently falling back to processing without cache. Default: `false`
+
+### Set up credentials
+
+There are three ways to specify your AWS credentials. The credentials need to have read/write rights for the cache bucket:
+
+#### IAM Roles
+
+If you're running imgproxy on an Amazon Web Services platform, you can use IAM roles to get the security credentials to make calls to AWS S3.
+
+* **Elastic Container Service (ECS):** Assign an [IAM role to a task](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html).
+* **Elastic Kubernetes Service (EKS):** Assign a [service account to a pod](https://docs.aws.amazon.com/eks/latest/userguide/pod-configuration.html).
+* **Elastic Beanstalk:** Assign an [IAM role to an instance](https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/iam-instanceprofile.html).
+
+#### Environment variables
+
+You can specify an AWS Access Key ID and a Secret Access Key by setting the standard `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables.
+
+```bash
+AWS_ACCESS_KEY_ID=my_access_key AWS_SECRET_ACCESS_KEY=my_secret_key imgproxy
+
+# same for Docker
+docker run -e AWS_ACCESS_KEY_ID=my_access_key -e AWS_SECRET_ACCESS_KEY=my_secret_key -it ghcr.io/imgproxy/imgproxy
+```
+
+#### Shared credentials file
+
+Alternatively, you can create the `.aws/credentials` file in your home directory with the following content:
+
+```ini
+[default]
+aws_access_key_id = %access_key_id
+aws_secret_access_key = %secret_access_key
+```
+
+#### Cross-Account Access
+
+Cache S3 access credentials may be acquired by assuming a role using STS. To do so, specify the IAM Role ARN with the `IMGPROXY_CACHE_S3_ASSUME_ROLE_ARN` environment variable. Additionally, if you require an external ID to be passed when assuming a role, specify the `IMGPROXY_CACHE_S3_ASSUME_ROLE_EXTERNAL_ID` environment variable. This approach still requires you to provide initial AWS credentials by using one of the ways described above. The provided credentials role should allow assuming the role with the provided ARN.
+
+### Choosing the AWS region
+
+The AWS region specified with the `IMGPROXY_CACHE_S3_REGION` environment variable determines the S3 endpoint used by imgproxy for the initial request to the bucket. If AWS reports that the bucket is in a different region, imgproxy will remember this, retry the request, and use the new region for all subsequent requests for this bucket.
+
+This allows imgproxy to access buckets in any region. However, the initial request to the bucket in a region other than the one specified in the environment variable may introduce some latency.