diff --git a/docs-site/content/26.0/api/cluster-operations.md b/docs-site/content/26.0/api/cluster-operations.md index 9fcf2439..8296de55 100644 --- a/docs-site/content/26.0/api/cluster-operations.md +++ b/docs-site/content/26.0/api/cluster-operations.md @@ -442,9 +442,22 @@ curl "http://localhost:8108/health" -When a node is running out of memory / disk, the API response will have an additional `resource_error` field that's +When a node is running out of memory / disk, the API response will have an additional `resource_error` field that's set to either `OUT_OF_DISK` or `OUT_OF_MEMORY`. +The `OUT_OF_MEMORY` error is triggered when the `--memory-used-max-percentage` server parameter is set to less than `100` +and free memory falls below the calculated threshold. The threshold is determined as: +`min(500MB, (100 - memory_used_max_percentage)% × total_memory)`. + +For example, with `--memory-used-max-percentage=90` on an 8GB system: +- Percentage-based threshold: (100-90)% × 8GB = 800MB free memory required +- Final threshold: min(500MB, 800MB) = 500MB +- Writes are rejected when free memory drops below 500MB + +When `--memory-used-max-percentage=100` (default), the resource check is skipped entirely and no memory-based rejection occurs. + +If `--memory-used-max-percentage` is below 100 and the threshold is reached, Typesense will reject write operations to prevent system instability. + #### Definition `GET ${TYPESENSE_HOST}/health` diff --git a/docs-site/content/27.0/api/cluster-operations.md b/docs-site/content/27.0/api/cluster-operations.md index 9fcf2439..8296de55 100644 --- a/docs-site/content/27.0/api/cluster-operations.md +++ b/docs-site/content/27.0/api/cluster-operations.md @@ -442,9 +442,22 @@ curl "http://localhost:8108/health" -When a node is running out of memory / disk, the API response will have an additional `resource_error` field that's +When a node is running out of memory / disk, the API response will have an additional `resource_error` field that's set to either `OUT_OF_DISK` or `OUT_OF_MEMORY`. +The `OUT_OF_MEMORY` error is triggered when the `--memory-used-max-percentage` server parameter is set to less than `100` +and free memory falls below the calculated threshold. The threshold is determined as: +`min(500MB, (100 - memory_used_max_percentage)% × total_memory)`. + +For example, with `--memory-used-max-percentage=90` on an 8GB system: +- Percentage-based threshold: (100-90)% × 8GB = 800MB free memory required +- Final threshold: min(500MB, 800MB) = 500MB +- Writes are rejected when free memory drops below 500MB + +When `--memory-used-max-percentage=100` (default), the resource check is skipped entirely and no memory-based rejection occurs. + +If `--memory-used-max-percentage` is below 100 and the threshold is reached, Typesense will reject write operations to prevent system instability. + #### Definition `GET ${TYPESENSE_HOST}/health` diff --git a/docs-site/content/27.1/api/cluster-operations.md b/docs-site/content/27.1/api/cluster-operations.md index 0ce2406e..5e597b07 100644 --- a/docs-site/content/27.1/api/cluster-operations.md +++ b/docs-site/content/27.1/api/cluster-operations.md @@ -477,6 +477,19 @@ curl "http://localhost:8108/health" When a node is running out of memory / disk, the API response will have an additional `resource_error` field that's set to either `OUT_OF_DISK` or `OUT_OF_MEMORY`. +The `OUT_OF_MEMORY` error is triggered when the `--memory-used-max-percentage` server parameter is set to less than `100` +and free memory falls below the calculated threshold. The threshold is determined as: +`min(500MB, (100 - memory_used_max_percentage)% × total_memory)`. + +For example, with `--memory-used-max-percentage=90` on an 8GB system: +- Percentage-based threshold: (100-90)% × 8GB = 800MB free memory required +- Final threshold: min(500MB, 800MB) = 500MB +- Writes are rejected when free memory drops below 500MB + +When `--memory-used-max-percentage=100` (default), the resource check is skipped entirely and no memory-based rejection occurs. + +If `--memory-used-max-percentage` is below 100 and the threshold is reached, Typesense will reject write operations to prevent system instability. + #### Definition `GET ${TYPESENSE_HOST}/health` diff --git a/docs-site/content/28.0/api/cluster-operations.md b/docs-site/content/28.0/api/cluster-operations.md index 2cf5d10a..3d1c9a66 100644 --- a/docs-site/content/28.0/api/cluster-operations.md +++ b/docs-site/content/28.0/api/cluster-operations.md @@ -477,6 +477,19 @@ curl "http://localhost:8108/health" When a node is running out of memory / disk, the API response will have an additional `resource_error` field that's set to either `OUT_OF_DISK` or `OUT_OF_MEMORY`. +The `OUT_OF_MEMORY` error is triggered when the `--memory-used-max-percentage` server parameter is set to less than `100` +and free memory falls below the calculated threshold. The threshold is determined as: +`min(500MB, (100 - memory_used_max_percentage)% × total_memory)`. + +For example, with `--memory-used-max-percentage=90` on an 8GB system: +- Percentage-based threshold: (100-90)% × 8GB = 800MB free memory required +- Final threshold: min(500MB, 800MB) = 500MB +- Writes are rejected when free memory drops below 500MB + +When `--memory-used-max-percentage=100` (default), the resource check is skipped entirely and no memory-based rejection occurs. + +If `--memory-used-max-percentage` is below 100 and the threshold is reached, Typesense will reject write operations to prevent system instability. + #### Definition `GET ${TYPESENSE_HOST}/health` diff --git a/docs-site/content/29.0/api/cluster-operations.md b/docs-site/content/29.0/api/cluster-operations.md index 2186bb11..6b67441a 100644 --- a/docs-site/content/29.0/api/cluster-operations.md +++ b/docs-site/content/29.0/api/cluster-operations.md @@ -477,6 +477,19 @@ curl "http://localhost:8108/health" When a node is running out of memory / disk, the API response will have an additional `resource_error` field that's set to either `OUT_OF_DISK` or `OUT_OF_MEMORY`. +The `OUT_OF_MEMORY` error is triggered when the `--memory-used-max-percentage` server parameter is set to less than `100` +and free memory falls below the calculated threshold. The threshold is determined as: +`min(500MB, (100 - memory_used_max_percentage)% × total_memory)`. + +For example, with `--memory-used-max-percentage=90` on an 8GB system: +- Percentage-based threshold: (100-90)% × 8GB = 800MB free memory required +- Final threshold: min(500MB, 800MB) = 500MB +- Writes are rejected when free memory drops below 500MB + +When `--memory-used-max-percentage=100` (default), the resource check is skipped entirely and no memory-based rejection occurs. + +When this condition is met, Typesense will reject write operations to prevent system instability. + #### Definition `GET ${TYPESENSE_HOST}/health` diff --git a/docs-site/content/29.0/api/server-configuration.md b/docs-site/content/29.0/api/server-configuration.md index 1a24e78f..80c70b1a 100644 --- a/docs-site/content/29.0/api/server-configuration.md +++ b/docs-site/content/29.0/api/server-configuration.md @@ -88,7 +88,7 @@ Command line arguments can be passed to the server as `--parameter=value`. | `--cache-num-entries` | false | Number of entries to be stored in the LRU cache used for storing search query responses. Default: `1000`. | | `--embedding-cache-num-entries` | false | Number of entries to be stored in the LRU cache used for storing text embeddings returned by remote embedders. Default: `100`. | | `--disk-used-max-percentage` | false | Reject writes when used disk space exceeds this percentage. Default: `100` (never reject). | -| `--memory-used-max-percentage` | false | Reject writes when used memory usage exceeds this percentage. Default: `100` (never reject). | +| `--memory-used-max-percentage` | false | Reject writes when free memory falls below a threshold. The threshold is calculated as the minimum of 500MB or `(100 - this_percentage)% × total_memory`. This will trigger an `OUT_OF_MEMORY` error. Default: `100` (resource check is skipped entirely, no rejection occurs). | | `--healthy-read-lag` | false | Reads are rejected if the updates lag behind this threshold. Default: `1000`. | | `--healthy-write-lag` | false | Writes are rejected if the updates lag behind this threshold. Default: `500`. | | `--snapshot-interval-seconds` | false | Frequency of replication log snapshots. Default: `3600` follower recovery.

**NOTE**: Frequent snapshotting helps in faster recovery from a cold start. However, if this value is too low for a large dataset, repeated snapshotting can actually slow down follower recovery. | diff --git a/docs-site/content/30.0/api/cluster-operations.md b/docs-site/content/30.0/api/cluster-operations.md index 2186bb11..f2f0a0fa 100644 --- a/docs-site/content/30.0/api/cluster-operations.md +++ b/docs-site/content/30.0/api/cluster-operations.md @@ -477,6 +477,19 @@ curl "http://localhost:8108/health" When a node is running out of memory / disk, the API response will have an additional `resource_error` field that's set to either `OUT_OF_DISK` or `OUT_OF_MEMORY`. +The `OUT_OF_MEMORY` error is triggered when the `--memory-used-max-percentage` server parameter is set to less than `100` +and free memory falls below the calculated threshold. The threshold is determined as: +`min(500MB, (100 - memory_used_max_percentage)% × total_memory)`. + +For example, with `--memory-used-max-percentage=90` on an 8GB system: +- Percentage-based threshold: (100-90)% × 8GB = 800MB free memory required +- Final threshold: min(500MB, 800MB) = 500MB +- Writes are rejected when free memory drops below 500MB + +When `--memory-used-max-percentage=100` (default), the resource check is skipped entirely and no memory-based rejection occurs. + +If `--memory-used-max-percentage` is below 100 and the threshold is reached, Typesense will reject write operations to prevent system instability. + #### Definition `GET ${TYPESENSE_HOST}/health` diff --git a/docs-site/content/30.0/api/server-configuration.md b/docs-site/content/30.0/api/server-configuration.md index f1aee369..70490f04 100644 --- a/docs-site/content/30.0/api/server-configuration.md +++ b/docs-site/content/30.0/api/server-configuration.md @@ -88,7 +88,7 @@ Command line arguments can be passed to the server as `--parameter=value`. | `--cache-num-entries` | false | Number of entries to be stored in the LRU cache used for storing search query responses. Default: `1000`. | | `--embedding-cache-num-entries` | false | Number of entries to be stored in the LRU cache used for storing text embeddings returned by remote embedders. Default: `100`. | | `--disk-used-max-percentage` | false | Reject writes when used disk space exceeds this percentage. Default: `100` (never reject). | -| `--memory-used-max-percentage` | false | Reject writes when used memory usage exceeds this percentage. Default: `100` (never reject). | +| `--memory-used-max-percentage` | false | Reject writes when free memory falls below a threshold. The threshold is calculated as the minimum of 500MB or `(100 - this_percentage)% × total_memory`. This will trigger an `OUT_OF_MEMORY` error. Default: `100` (resource check is skipped entirely, no rejection occurs). | | `--healthy-read-lag` | false | Reads are rejected if the updates lag behind this threshold. Default: `1000`. | | `--healthy-write-lag` | false | Writes are rejected if the updates lag behind this threshold. Default: `500`. | | `--snapshot-interval-seconds` | false | Frequency of replication log snapshots. Default: `3600` follower recovery.

**NOTE**: Frequent snapshotting helps in faster recovery from a cold start. However, if this value is too low for a large dataset, repeated snapshotting can actually slow down follower recovery. |