SGBox Log Management and Retention

This section explain how SGBox manage logs and difference between: online logs, encrypted logs and events.

Requirements:

• SGBox version 5.0.4 is required.

From the beginning: SGBox receives log data from different inputs / data sources and by using different protocols.

+Online Logs
The raw data from each data source is associated with a unique tag and placed in a local storage to SGBox. In this position, which we will call the “online log repository”, the data is kept in a compressed and indexed format, so that the user can consult it at any time through the Historical search ( SCM -> Analysis -> Historical search ).
Online data cannot be modified in any way by the user and at regular intervals data is extracted, divided by data source and placed in another storage, not accessible to the user, in encrypted format.

+Encrypted Logs
Data present in the “encrypted log data storage” are encrypted using GPG standard and cannot be modified in any way by the user. Each encrypted block is also compressed, signed and associated with a timestamp linked to the encryption key.
User is always able to verity the data integrity of the encrypted data by using a dedicated SGBox view ( LM -> Configuration -> Encryption ).

The only operations that the user can perform on this data are
1. define a retention of the data that can be different for each data source ( SCM -> Advanced Options or LM -> Network -> Hosts ).
2. transfer a copy of the data to online storage ( LM -> Configuration -> Online logs manager SGBox vers. > 5.0.4 )
3. backup the data using applications (SCM -> Applications ).

+Events
While data is collected, it is also analyzed by a series of parsers capable of extracting meaningful information from raw data, transforming it into what SGBox calls “events”.
Events, just like online logs, are freely accessible by the user. Events generated by log parsing, report the same unique id the logs were tagged with, to definitively bind the event and the log that generated it. User can rebuild events, enriching them with new information, build correlation rules and dashboards on them, while the algorithms – always active on SGBox – analyze events searching for anomalies in behaviors and volumes.

This figure shows the log storage architecture and the access rules to the online log and events data (Remember: only online logs and events can be accessed by user)

Note that data exported to the external storage always leaves SGBox in encrypted and compressed form.
When restoring data from the external storage you can always check for data consistency by validating the signature for each block of data.

You can set the retention from SCM > Advanced Options

In the Retention section