Skip to content

s3: update retry_limit default to 5 #2536

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
singholt wants to merge 1 commit into
base: master
Choose a base branch
from
Open

s3: update retry_limit default to 5 #2536

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion pipeline/outputs/s3.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ The [Prometheus success/retry/error metrics values](../../administration/monitor
| `preserve_data_ordering` | When an upload request fails, the last received chunk might swap with a later chunk, resulting in data shuffling. This feature prevents shuffling by using a queue logic for uploads. | `true` |
| `profile` | Option to specify an AWS Profile for credentials. | _none_ |
| `region` | The AWS region of your S3 bucket. China regions (`cn-north-1`, `cn-northwest-1`) and the AWS European Sovereign Cloud region (`eusc-de-east-1`) are supported; Fluent Bit automatically uses the correct endpoint suffix (`.amazonaws.com.cn` or `.amazonaws.eu`) and no custom `endpoint` is required. | `us-east-1` |
| `retry_limit` | Set retry limit for output plugin when delivery fails. Integer, `no_limits`, `false`, or `off` to disable, or `no_retries` to disable retries entirely. | `1` |
| `retry_limit` | Set the maximum number of retries for the S3 plugin's internal retry system. Unlike other output plugins, S3 manages its own buffering and retries internally. If not explicitly set, defaults to `5` to allow sufficient retries for transient S3 errors and avoid wasting partially uploaded multipart data. Accepts an integer or `no_retries` to disable retries entirely. | `5` |
| `role_arn` | ARN of an IAM role to assume (for example, for cross account access). | _none_ |
| `s3_key_format` | Format string for keys in S3. This option supports a UUID, strftime time formatters, a syntax for selecting parts of the Fluent log tag using a syntax inspired by the `rewrite_tag` filter. Add `$UUID` in the format string to insert a random string. Add `$INDEX` in the format string to insert an integer that increments each upload. The `$INDEX` value saves in the `store_dir`. Add `$TAG` in the format string to insert the full log tag. Add `$TAG[0]` to insert the first part of the tag in the S3 key. The tag is split into parts using the characters specified with the `s3_key_format_tag_delimiters` option. Add the extension directly after the last piece of the format string to insert a key suffix. To specify a key suffix in `use_put_object` mode, you must specify `$UUID`. See [S3 Key Format](#s3-key-format-and-tag-delimiters). Time in `s3_key` is the timestamp of the first record in the S3 file. | `/fluent-bit-logs/$TAG/%Y/%m/%d/%H/%M/%S` |
| `s3_key_format_tag_delimiters` | A series of characters which will be used to split the tag into `parts` for use with the s3_key_format option. | `.` |
Expand Down
Loading