logs disappearing from kibana

[ebdavison]

Version

2.4.100

Installation Method

Security Onion ISO image

Description

other (please provide detail below)

Installation Type

Standalone

Location

cloud

Hardware Specs

Meets minimum requirements

CPU

6

RAM

16GB

Storage for /

588G

Storage for /nsm

588G ( part of / )

Network Traffic Collection

other (please provide detail below)

Network Traffic Speeds

1Gbps to 10Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

No, there are no failures

Logs

Yes, there are additional clues in /opt/so/log/ (please provide detail below)

Detail

I have firewall logs that I need to keep around and not have them be deleted as these are critical for incident response and customer reporting. When checking the logs I see:

Thu Sep 19 07:40:05 PM UTC 2024 - Used disk space exceeds LOG_SIZE_LIMIT (294 GB) - There is only one backing index (.ds-logs-fortinet_fortigate.log-default-2024.09.19-000001).  Deleting logs-fortinet_fortigate.log-default data stream...

How can I prevent these fortigate logs from being deleted?

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[kvkamin]

Disk space is managed by retention settings and available space. You have 588GB which is probably not enough for your use case. You have to decide how long you need to keep those records and figure out how much space that takes. You may need multiple TB.

[ebdavison]

Sounds reasonable. Where are the retention settings configured?

I notice that there are over 6 months worth (> 25 Million) of syslog docs but the fortigate logs have gotten deleted overnight (and somewhat regularly for past several months). All of the syslog entries are coming from the SO system itself and have not been being deleted.

How can I flip this and prioritize the foritgate logs and start rolling the syslog logs off after 90 days?

[dougburks]

Have you checked the Elasticsearch section of our documentation?
https://docs.securityonion.net/en/2.4/elasticsearch.html#index-management

Since your node is standalone, much of your disk space will be used by full packet capture, so you may also want to review the relevant sections of the documentation and consider tuning your PCAP space. You are most likely using Stenographer for PCAP unless you've manually switched to Suricata PCAP:
https://docs.securityonion.net/en/2.4/stenographer.html#disk-free-percentage
https://docs.securityonion.net/en/2.4/suricata.html#pcap

However, as mentioned above, 588GB is not that much for a production Security Onion deployment so you may want to consider increasing your storage.

[ebdavison]

Well, this is a cloud install so there are no packet captures taking place. That is not likely to be filling the disk.

I will review the docs linked above.