Skip to main content

Overview

What are structured logs​

Structured logs contain protobuf based time series data with details of the operation of the solution.

They exist as many streams of data, each containing similar protobuf data (in terms of schema and use case). We refer to each of these streams by the name "event source". Some examples of this data are pose estimation results, robot telemetry while moving, and robot transforms.

tip

Structured logs are different from text logs in the sense that every logged item is:

  • a data structure rather than an unstructured text string, and
  • includes metadata about the source and
  • includes context information which allows log messages to be nested to indicate their relationship.

Data structure​

The central data structure for structured logging is the LogItem. It consists of:

  • metadata (which names the event source),
  • a context (which allows log messages to relate to each other),
  • a payload (a well known proto, e.g., an error report, the executive state or skill requests/results), and;
  • (optionally) a blob (opaque bytes array, e.g., meshes or images)

Data flow​

Structured logs are captured as they are logged in the IPC, and selectively uploaded to the cloud data infrastructure. This allows for remote analysis and debugging.

The upload to the cloud is dependent on the available internet connectivity. If connectivity is limited, only a subset of structured logs are sent to the cloud on a best-effort basis.

In addition to the best-effort upload to the cloud, there is a way to prioritize a time range of structured logs to be robustly uploaded to the cloud as a solution recording.

Storage and retention​

Mirroring the data flow above, structured logs are stored in and are accesible from a few locations.

  • On the IPC: Retained for a period of 24 hours by default
  • On the cloud: Retained for a period of 14 days
  • On the cloud, as recordings: Retained for a period of 30 days