Sometimes the application insert some junk information that are not useful in the logs.
Here a description on how to collect the important information:

In our example there are some values like: current_link, next_link and last_log_item_generation that are excluded

Last start key: means the array where our values are stored, here an example of the scructure:

"security_events":[{event1}, {event2}, ... {eventN}]

Timestamp key: means the key that indicate the timestamp. in our example we put message.detection_time because detection_time is a key nested in message

Timestamp format: means the format of our timestamp.

Logs nodes allows you to take an input, usually coming from a request to an API to retrieve logs, and process it to extract and store to SGBox a set of log lines.
The input always comes from a previous node in the flow. Select a node from the list to show its output.
Available nodes, to manage different input formats, are

• JSON logs
• CSV logs
• TEXT logs

JSON logs

Once the JSON output of a previous node is displayed, you can click on its keys to tell the node where to extract the logs from. Then, you have to tell the node where to extract the timestamp from, its format, the host, and other info. Follow the guide inside the node form.

See this article if you there are some problem to collect the logs see the advanced section

CSV logs

Retrieve logs from API

You can create a Playbook with one or more api requests, using Start timestamp and End timestamp parameters, followed by a Logs node.
Then, just schedule the playbook to be executed with any periodicity and, at every execution, it will call the API with updated timestamps to retrieve the last available logs and store them into SGBox, available for consultation in the Historical Search page.

Nodes allows you to extract, from an input, a list of values to be stored into SGBox, when the playbook is associated with a feed.
The input always comes from a previous node in the flow. Select a node from the list to show its output.
Available nodes, to manage different input formats, are

• List from JSON
• List from CSV
• List from TEXT

List from JSON

Once the JSON output of a previous node is displayed, you can click on its keys to tell the node where to extract the list from. Follow the guide inside the node form.

List from CSV