Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIX: Intermittent soup errors causing soup to exit with failure message #13247

Closed
jertel opened this issue Jun 21, 2024 · 1 comment
Closed
Assignees
Milestone

Comments

@jertel
Copy link
Contributor

jertel commented Jun 21, 2024

  1. When shutting down salt-master, if it exits before the tail --pid ... command starts then the tail will error out and a message is echo'd to console that the salt-master did not shutdown within 30 seconds. That's not correct and should not be printed out in that situation.
    Ex:
tail: invalid PID: ''
salt-master still running at 10:13:25.600201 after waiting 30s. We cannot kill due to systemd restart option.
  1. Soup needs to set +e before upgrading the elastic agent and then reset back to set -e after it returns. This will prevent soup from exiting if the first curl fails (it's in a retry loop).
    Ex:
Checking to see if changes are needed.
parse error: Invalid numeric literal at line 1, column 7
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    68  100    30  100    38   1917   2Kibana server is not ready yet428 --:--:-- --:--:-- --:--:--  4533
Checking if Elastic Agent update is necessary...
Executing command with retry support: curl --fail --retry 5 --retry-delay 15 -L 'https://repo.security((redacted)).net/file/so-repo/prod/2.4/elasticagent/elastic-agent_SO-8.10.4.tar.gz' --output '/nsm/elastic-fleet/artifacts/elastic-agent_SO-8.10.4.tar.gz'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
 55 1389M   55  778M    0     0  10.2M      0  0:02:15  0:01:15  0:01:00  7597
curl: (92) HTTP/2 stream 0 was not closed cleanly: INTERNAL_ERROR (err 2)

It's TBD where the parse error originated in this example.

@jertel jertel added this to the 2.4.90 milestone Jun 21, 2024
@jertel jertel changed the title FIX: Correct intermittent soup errors FIX: Intermittent soup errors causing soup to exit with failure message Jun 21, 2024
@TOoSmOotH TOoSmOotH modified the milestones: 2.4.90, 2.4.100 Jul 17, 2024
@defensivedepth defensivedepth modified the milestones: 2.4.100, 2.4.110 Aug 26, 2024
@m0duspwnens m0duspwnens self-assigned this Sep 16, 2024
@m0duspwnens
Copy link
Contributor

Executing command with retry support: curl --fail --retry 5 --retry-delay 15 -L 'https://repo.securityonion.net/file/so-repo/prod/2.4/elasticagent/elastic-agent_SO-8.14.3.tar.gz' --output '/nsm/elastic-fleet/artifacts/elastic-agent_SO-8.14.3.tar.gz'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:02:09 --:--:--     0
curl: (28) Failed to connect to repo.securityonion.net port 443: Connection timed out
Warning: Problem : timeout. Will retry in 15 seconds. 5 retries left.
  0     0    0     0    0     0      0      0 --:--:--  0:02:08 --:--:--     0
curl: (28) Failed to connect to repo.securityonion.net port 443: Connection timed out
Warning: Problem : timeout. Will retry in 15 seconds. 4 retries left.
100  851M  100  851M    0     0  38.2M      0  0:00:22  0:00:22 --:--:-- 43.6M
Results:  (0)
Executing command with retry support: curl --fail --retry 5 --retry-delay 15 -L 'https://repo.securityonion.net/file/so-repo/prod/2.4/elasticagent/elastic-agent_SO-8.14.3.md5' --output '/nsm/elastic-fleet/artifacts/elastic-agent_SO-8.14.3.md5'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    33  100    33    0     0     78      0 --:--:-- --:--:-- --:--:--    78
Results:  (0)
Source file and checksum are good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants