Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions blog-service/2026-07-09-search.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
title: Increased Maximum Log Message Size to 256KB (Search)
image: https://assets-www.sumologic.com/company-logos/_800x418_crop_center-center_82_none/SumoLogic_Preview_600x600.jpg?mtime=1617040082
keywords:
- log message size
- large messages
- ingestion
hide_table_of_contents: true
---

We're excited to announce that Sumo Logic now supports a maximum log message size of **256KB**, up from 64KB, so large single-line logs such as AWS CloudTrail, Google Cloud, and Microsoft 365 events are ingested without being split as often. Some features still handle messages at 64KB, including LogCompare, LogReduce, Cloud SIEM parsing, and Field Extraction Rules. [Learn more](/docs/search/get-started-with-search/search-basics/search-large-messages/).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AWS CloudTrail, Google Cloud, and Microsoft 365 events - are they known large log sources?

2 changes: 1 addition & 1 deletion docs/reuse/fer-limitations.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,5 @@ Ingest Time FERs have the following limitations:
:::
* Ingest Time rule expressions are limited to a maximum of 16k (16,384) characters.
* Ingest Time rules can extract up to a maximum of 16k (16,384) characters for each field.
* The cumulative size of all fields extracted by a rule for a message/event is limited to 64kb.
* The cumulative size of all fields extracted by a rule for a message/event is limited to 64kb, regardless of the maximum log message size.
* Ingest Time rules **only apply to data moving forward**. If you want to [parse](/docs/search/search-query-language/parse-operators) data ingested before the creation of your Ingest Time FER, you can either parse your data in your query, or create [Scheduled Views](/docs/manage/scheduled-views) to extract fields for your historical data.
4 changes: 4 additions & 0 deletions docs/search/behavior-insights/logcompare.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ LogCompare allows you to easily compare log data from different time periods to
For example, you could determine what was different right before a failure compared to the previous day or previous week. Or, you could easily check if a new release introduced a new issue by reviewing the difference in log streams across time.


:::note
LogCompare truncates raw 256KB messages to 64KB before matching and grouping the logs into signatures, so content beyond 64KB is not compared. This can also affect response time when you run LogCompare against large messages.
:::

## How LogCompare works

LogCompare creates a baseline (historical) query and compares it to a target (current) query. Logs from each query are clustered into patterns, called signatures, then compared and ranked based on the significance of change.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ The **Signatures** tab displays LogReduce results as signatures. A signature

You can refine the results of the LogReduce algorithm to make the outcome more generic or more specific. See [Influencing the LogReduce Outcome](influence-the-logreduce-outcome.md) for more information.

:::note
LogReduce truncates raw 256KB messages to 64KB before matching and grouping the logs into signatures, so content beyond 64KB is not included. This can also affect response time when you run LogReduce against large messages.
:::

## Will my LogReduce search results match my keyword search results?

Generally speaking, no. LogReduce is intended to be a jumping-off point for your analysis. Unlike a keyword search, where you are looking for data related to, say, a specific source or an error message, LogReduce returns signatures that contain messages that *may* be of interest to you using fuzzy logic. If you are not happy with a signature, you can [teach](influence-the-logreduce-outcome.md) LogReduce how you'd like the results to be made more specific. do not think of a signature as an example of what logs are grouped under it; instead think of a signature as a reflection of what LogReduce thinks you will find interesting if that signature catches your eye. Once you begin digging in to LogReduce results, you will then want to structure a keyword query that delivers precise results.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
---
id: search-large-messages
title: Search Large Messages
description: When collecting log messages or event logs that are larger than 64KB in size, Sumo Logic slices the messages into a stream of smaller message chunks.
description: When collecting log messages or event logs that are larger than 256KB in size, Sumo Logic slices the messages into a stream of smaller message chunks.
---



When collecting log messages or event logs that are larger than 64KB in size, Sumo Logic slices the messages into a stream of smaller message chunks.
When collecting log messages or event logs that are larger than 256KB in size, Sumo Logic slices the messages into a stream of smaller message chunks.

Chunks are ideally created at a line break depending on the Source type, protocol, and size of the message. Each section of the large messages is annotated with metadata to keep the message in order when viewing or searching the log.

Expand All @@ -16,8 +16,17 @@ For additional information on collecting large messages review the relevant [Sou

## Query to Identify Truncated Logs 

We have a [metadata](built-in-metadata.md) tag called `_size`. The `_size` metadata tag provides the size of the log message in bytes. A log is truncated at the size of 64K or 65536 bytes.
We have a [metadata](built-in-metadata.md) tag called `_size`. The `_size` metadata tag provides the size of the log message in bytes. A log is truncated at the size of 256K or 262144 bytes.

To find truncated logs you can reference the `_size` metadata tag, an example would be using the where operator.

`| where _size = 65536`
`| where _size = 262144`

## Known limitations

Keep the following limitations in mind when you work with messages larger than 64KB:

- **LogCompare and LogReduce**. These operators truncate raw 256KB messages to 64KB before matching and grouping the logs into signatures, so content beyond 64KB is not considered. This can also affect response time when you run them against large messages. Learn more in [LogReduce](/docs/search/behavior-insights/logreduce/) and [LogCompare](/docs/search/behavior-insights/logcompare/).
- **Log Search messages table**. When you view large messages, the messages table displays up to 25 messages per page.

@lei-sumo lei-sumo Jul 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This 25 msg/page UI limitation will be removed as part of the GA based on this comment: https://sumologic.slack.com/archives/C02DUQT0A/p1783604422222719?thread_ts=1777391223.682879&cid=C02DUQT0A

The new behavior is that search page will only show upto 64kb in the results table (after expansion), but the complete individual message can be viewed in the message inspector

- **Cloud SIEM**. Parsing and mapping might not process messages larger than 64KB correctly.
- **Field Extraction Rules**. The cumulative size of all fields extracted by a rule for a message is limited to 64KB, regardless of the message size. Learn more in [Field Extraction Rule limitations](/docs/manage/field-extractions/create-field-extraction-rule/#limitations).
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import useBaseUrl from '@docusaurus/useBaseUrl';

You can configure a cloud syslog source to allow a syslog client to send [RFC 5424](https://tools.ietf.org/html/rfc5424)-compliant messages to Sumo Logic. A source-specific token, generated by Sumo Logic, is inserted into each message to identify the source. Transport Layer Security (TLS) 1.3 over TCP is required.

Syslog messages must be compliant with [RFC 5424](https://tools.ietf.org/html/rfc5424) or they are dropped. Messages over 64 KB are truncated.
Syslog messages must be compliant with [RFC 5424](https://tools.ietf.org/html/rfc5424) or they are dropped. Messages over 256 KB are truncated.

Sumo Logic manages an elastic scaling set of syslog servers, which scales up and down behind a set of AWS Elastic Load Balancers. The AWS ELB set can also scale up and down. For this reason, instead of IP address-based endpoints, Sumo Logic uses endpoint hostnames in this format:

Expand Down Expand Up @@ -103,7 +103,7 @@ Follow the configuration instructions for your syslog client:

### Message format

Syslog messages must be in RFC 5424-compliant form or they are dropped. Messages over 64KB in length are truncated.
Syslog messages must be in RFC 5424-compliant form or they are dropped. Messages over 256KB in length are truncated.

This diagram shows the RFC 5424 format:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ Click [here](/docs/c2c/info) for more information about Cloud-to-Cloud sources.

### Is anything changed with data for computer inventory?

Sometimes the asset information from the computer inventory data can exceed the [Sumo Logic maximum log size of 64KB](/docs/search/get-started-with-search/search-basics/search-large-messages/). Sumo Logic will automatically split log messages exceeding the size limit into smaller chunks. This C2C makes the following changes to the computer inventory asset data collected in order to keep most logs under the size limit and prevent splitting:
Sometimes the asset information from the computer inventory data can exceed the [Sumo Logic maximum log size of 256KB](/docs/search/get-started-with-search/search-basics/search-large-messages/). Sumo Logic will automatically split log messages exceeding the size limit into smaller chunks. This C2C makes the following changes to the computer inventory asset data collected in order to keep most logs under the size limit and prevent splitting:

- The `openPortListData` key only contains information about ports open since the last time computer asset was ingested instead of listing all open port history from all time.
- The `softwareListData` is reduced down from the full details to simply a list/array of software names using the full name.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ The line breaks or carriage returns are control characters used to create new li
Multiline messages that are more than 2,000 lines or 512KB in size will get flushed and collected as single log lines due to the default log message size limitations. Depending on the Collector's available memory, you may be able to increase this limit. Contact Support for assistance by navigating to **Help** > **Support** in the Sumo Logic
menu.

Messages will be further broken down when they are received and indexed. When collecting log messages that are larger than 64KB in size, Sumo Logic slices the messages into a stream of smaller message chunks. Chunks are ideally created at a line break depending on the Source type, protocol, and size of the message. Each section of the large messages is annotated with metadata to keep the message in order when viewing or searching the log. Review the relevant Source's documentation for additional information.
Messages will be further broken down when they are received and indexed. When collecting log messages that are larger than 256KB in size, Sumo Logic slices the messages into a stream of smaller message chunks. Chunks are ideally created at a line break depending on the Source type, protocol, and size of the message. Each section of the large messages is annotated with metadata to keep the message in order when viewing or searching the log. Review the relevant Source's documentation for additional information.

## Multiline Processing Options

Expand Down