Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Cloud Security] [Agentless] Create GHA Cloud Security workflow sanity tests for Agentless ESS deployments. #192182

Open
wants to merge 21 commits into
base: main
Choose a base branch
from
Open

[Cloud Security] [Agentless] Create GHA Cloud Security workflow sanity tests for Agentless ESS deployments. #192182

wants to merge 21 commits into from

Conversation

seanrathier
Copy link
Contributor

@seanrathier seanrathier commented Sep 5, 2024 •
edited
Loading

Summary

We are adding agentless sanity FTR tests that will be executed when a developer creates a Coud Security deployment or project using our GHA in the CloudBeat repo.

These FTR tests are not running a UX experience of adding an integration, the Agentless agent integration will be added during the creation of the deployment or project.

Testing the workflow

To test this run the CreateEnvironment workflow with the following parameters

  • Workflow branch: 10463-create-agentless-agent
  • Stack version: 8.16.0-SNAPSHOT
  • Select Run UI sanity tests after provision checkbox
  • Kibana UI test branch: 10269-gha-sanity-tests-for-agentless-ess-deployments

Local Testing instructions

  1. Create an deployment, if it does not have the Kibana agentless config you can manually add them
  2. Create a CSPM integration using Agentless option
  3. Validate that the data has been received
  4. Run the tests using the following command line in kibana directory
export TEST_CLOUD=1
export ES_SECURITY_ENABLED=1

export TEST_KIBANA_PROTOCOL=https
export TEST_KIBANA_HOSTNAME=<your-deployment-name>.kb.us-west2.gcp.elastic-cloud.com
export TEST_KIBANA_PORT=443
export TEST_KIBANA_USER=kibana_system
export TEST_KIBANA_PASS=<password>

export TEST_ES_PROTOCOL=https
export TEST_ES_HOSTNAME=<your-deployment-name>.es.us-west2.gcp.elastic-cloud.com
export TEST_ES_PORT=443
export TEST_ES_USER=kibana_system
export TEST_ES_PASS=<password>

Related Issues

Checklist

@seanrathier seanrathier marked this pull request as ready for review September 6, 2024 18:30
@seanrathier seanrathier requested a review from a team as a code owner September 6, 2024 18:30
@seanrathier seanrathier self-assigned this Sep 6, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security)

@seanrathier seanrathier added automation 8.16 candidate release_note:skip Skip the PR/issue when compiling release notes backport:skip This commit does not require backporting labels Sep 6, 2024
@seanrathier seanrathier force-pushed the 10269-gha-sanity-tests-for-agentless-ess-deployments branch from b065a81 to cd65995 Compare September 6, 2024 18:48
gurevichdmitry
gurevichdmitry reviewed Sep 8, 2024
View reviewed changes
x-pack/test/cloud_security_posture_functional/cloud_tests/agentless_sanity.ts Outdated
]);

const CIS_AWS_OPTION_TEST_ID = 'cisAwsTestId';
describe('Agentless Cloud - Sanity Tests', function () {
describe('agentless agent health', function () {
Copy link
Contributor

@gurevichdmitry gurevichdmitry Sep 8, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if testing agent health through the UI is the most suitable approach for our case. The Fleet component isn't under our responsibility, and although you're using test objects for some parts, you're relying on CSS selectors and retrieving visible text for the verification steps, which is more critical, there's a risk that if the Fleet team changes something related to the UI, it could break our tests.

If we still want to pinpoint where our tests are failing—whether the issue is on the UI side or due to the agent's health—it might be better to use a Fleet API call to verify if the required agent is healthy.

In the next test suite, we are already querying agentless data. If that suite fails, it should be sufficient to help identify where the problem lies.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've since removed the health checks and decided to simplify this and only check that we receive findings. If we are receiving findings that should indicate a good health and working Agentless agent.

@seanrathier seanrathier force-pushed the 10269-gha-sanity-tests-for-agentless-ess-deployments branch 2 times, most recently from f8e226e to efec89d Compare September 12, 2024 19:23
seanrathier
seanrathier commented Sep 12, 2024
View reviewed changes
x-pack/test/cloud_security_posture_functional/cloud_tests/agentless_api_sanity.ts Outdated
const queryBar = getService('queryBar');
const pageObjects = getPageObjects(['common', 'header', 'cisAddIntegration', 'findings']);

// let cisIntegration: typeof pageObjects.cisAddIntegration;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The commented code in this file will be enabled after

@seanrathier seanrathier added backport:prev-minor Backport to the previous minor version (i.e. one version back from main) and removed backport:skip This commit does not require backporting labels Sep 16, 2024
@seanrathier seanrathier force-pushed the 10269-gha-sanity-tests-for-agentless-ess-deployments branch from efe7796 to 71a47a5 Compare September 20, 2024 21:47
@kibana-ci
Copy link
Collaborator

kibana-ci commented Sep 20, 2024 •
edited
Loading

💔 Build Failed

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #101 / dashboard app - group 1 Changing field formatter to Url applied on discover
  • [job] [logs] FTR Configs #101 / dashboard app - group 1 Changing field formatter to Url applied on discover
  • [job] [logs] FTR Configs #15 / discover/group6 discover time field column should add a time field column ESQL mode should render selected columns correctly
  • [job] [logs] FTR Configs #15 / discover/group6 discover time field column should add a time field column ESQL mode should render selected columns correctly
  • [job] [logs] FTR Configs #12 / lens app - group 6 lens workspace size gauge size (absolute pixels) - circle
  • [job] [logs] FTR Configs #12 / lens app - group 6 lens workspace size gauge size (absolute pixels) - circle
  • [job] [logs] FTR Configs #11 / Reporting APIs Network Policy "after all" hook for "should fail job when page violates the network policy"
  • [job] [logs] FTR Configs #11 / Reporting APIs Network Policy "after all" hook for "should fail job when page violates the network policy"
  • [job] [logs] FTR Configs #11 / Reporting APIs Network Policy should fail job when page violates the network policy
  • [job] [logs] FTR Configs #11 / Reporting APIs Network Policy should fail job when page violates the network policy

Metrics [docs]

‼️ ERROR: no builds found for mergeBase sha [d4ee1ca]

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @seanrathier

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gurevichdmitry gurevichdmitry Awaiting requested review from gurevichdmitry

At least 1 approving review is required to merge this pull request.

Assignees

@seanrathier seanrathier

Labels
8.16 candidate automation backport:prev-minor Backport to the previous minor version (i.e. one version back from main) release_note:skip Skip the PR/issue when compiling release notes Team:Cloud Security Cloud Security team related
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@seanrathier @elasticmachine @kibana-ci @gurevichdmitry