No new logs visible in elastic out of nowhere #15224
-
Version2.4.190 Installation MethodSecurity Onion ISO image Descriptionconfiguration Installation TypeDistributed Locationon-prem with Internet access Hardware SpecsExceeds minimum requirements CPU48 devided over the different machines RAM300gb devided over the different machines Storage for /300gb Storage for /nsm3tb Network Traffic Collectionother (please provide detail below) Network Traffic Speeds1Gbps to 10Gbps StatusYes, all services on all nodes are running OK Salt StatusNo, there are no failures LogsYes, there are additional clues in /opt/so/log/ (please provide detail below) DetailHi, We have a distributed setup with 1 manager, 1 forwarder node and 3 search nodes. They ran fine for 2 years, but for some reason the logging seemed to stopped yesterday for (so far I can see) no obvious reason. All the status pages show ok and green. the only thing I could find was some logs from logstash that started yesterday about an expired certificate, but the strange part is that I cannot find which certificate is expired (every certificate in /etc/pki/ is still valid) and if this is even the culprit of the situation. I cannot find that elastic is read-only, the status sometimes states pending I think because of relocation of shards. Sometimes elastalert is missing, but comes back automatically.
Guidelines
|
Beta Was this translation helpful? Give feedback.
Replies: 4 comments 3 replies
-
|
Could it be some sort of mutual TLS where the client package of the elastic agent that is deployed by security onion still has the old certificate? |
Beta Was this translation helpful? Give feedback.
-
|
EDIT: commands should run from the manager / managersearch You said all the certs are valid? We can recheck with something like (runs on each node that is part of the log ingest process) If all services are running can you try also checking elasticsearch status (and provide output) Have you done any customization to any logstash pipelines? |
Beta Was this translation helpful? Give feedback.
-
|
How are you ingesting logs? Via the Elastic Agent ? If you go to SOC -> Downloads and download a fresh Elastic Agent are you able to install it on a new host and receive logs? |
Beta Was this translation helpful? Give feedback.
-
|
This was handled by Security Onion Professional Services. Replaced the certificate and key in Fleet > Logstash output. |
Beta Was this translation helpful? Give feedback.

This was handled by Security Onion Professional Services.
Replaced the certificate and key in Fleet > Logstash output.