Handle long log entries with split logs

Cloud Logging limits the size on incoming logs to 256KB and drops anything bigger. To ensure that Cloud Logging retains your large logs, Fleet Engine can split them into a series of smaller logs.

Cloud logging may split the following logs from Fleet Engine:

Each split log entry contains the following fields:

  • split.uid: A unique identifier for the group of log entries that were split from a common original log entry. The value of this field is the same for all entries split from the original log entry.
  • split.index: The position of this entry in the series of split entries. The first entry from the split has index 0.split.index. This index is also appended to the LogEntry.insertId field.
  • split.totalSplits: The number of log entries that the original log entry was split into. The value of this field is the same for all entries split from the original log entry.
split log 1:
insertId: "XXXX-01"
split {index: 0, uuid: "XXXX"}

splitLog 2:
insertId: "XXX-02"
split {index: 1, uuid: "XXXX"}

To find all the logs that were split from one specific log, use a query like:

    split.uid="789+2022-02-22T12:22:22.22+05:00"
    sortby split.index OR sortby insertID

The structure of these split logs is almost the same as the structure shown in the guide for Cloud Audit Logs. The major difference is that for Fleet Engine logs, the split occurs in the jsonPayload field. For details and examples, see Split audit log entries.

What's next

To count and filter logs according to you criteria, create log-based metrics.