Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make test historical retrieval longer #1630

Merged

Conversation

MattDelac
Copy link
Collaborator

@MattDelac MattDelac commented Jun 8, 2021

What this PR does / why we need it:
In order to have confidence in merging #1602, we want to also benchmark the tests performing historical retrieval on "larger" datasets.
This will ensure that a change in the template SINGLE_FEATURE_VIEW_POINT_IN_TIME_JOIN won't slow down the operations

image

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

NONE

Make `test_historical_features_from_bigquery_sources()` executed on larger datasets

@feast-ci-bot
Copy link
Collaborator

Hi @MattDelac. Thanks for your PR.

I'm waiting for a feast-dev member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@MattDelac MattDelac force-pushed the make_test_historical_retrieval_longer branch 2 times, most recently from 83c03a7 to 8c569d3 Compare June 8, 2021 18:51
@codecov-commenter
Copy link

codecov-commenter commented Jun 8, 2021

Codecov Report

Merging #1630 (55b06c2) into master (17231d0) will decrease coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1630      +/-   ##
==========================================
- Coverage   83.88%   83.87%   -0.01%     
==========================================
  Files          67       67              
  Lines        5883     5911      +28     
==========================================
+ Hits         4935     4958      +23     
- Misses        948      953       +5     
Flag Coverage Δ
integrationtests 83.87% <100.00%> (+0.07%) ⬆️
unittests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
sdk/python/tests/test_historical_retrieval.py 98.93% <100.00%> (+0.04%) ⬆️
sdk/python/tests/conftest.py 83.87% <0.00%> (-16.13%) ⬇️
sdk/python/tests/test_init.py 100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 17231d0...55b06c2. Read the comment docs.

Signed-off-by: Matt Delacour <matt.delacour@shopify.com>
@MattDelac MattDelac force-pushed the make_test_historical_retrieval_longer branch from 8c569d3 to 55b06c2 Compare June 8, 2021 18:53
@feast-ci-bot feast-ci-bot added size/M and removed size/S labels Jun 8, 2021
@MattDelac
Copy link
Collaborator Author

/kind housekeeping

@MattDelac
Copy link
Collaborator Author

/ok-to-test

@feast-ci-bot
Copy link
Collaborator

@MattDelac: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/ok-to-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@achals
Copy link
Member

achals commented Jun 8, 2021

/ok-to-test

@achals
Copy link
Member

achals commented Jun 8, 2021

PASSED [ 66%]
sdk/python/tests/test_historical_retrieval.py::test_historical_features_from_bigquery_sources[False-gcp] 
Time to execute job_from_df.to_df() = '0:00:03.311531'
Time to execute job_from_df.to_df() = '0:00:03.335998'

PASSED [ 67%]
sdk/python/tests/test_historical_retrieval.py::test_historical_features_from_bigquery_sources[False-gcp_custom_offline_config] 
Time to execute job_from_df.to_df() = '0:00:03.019203'
Time to execute job_from_df.to_df() = '0:00:03.901726'

PASSED [ 68%]
sdk/python/tests/test_historical_retrieval.py::test_historical_features_from_bigquery_sources[True-local] 
Time to execute job_from_df.to_df() = '0:00:04.391762'
Time to execute job_from_df.to_df() = '0:00:02.871105'

PASSED [ 69%]
sdk/python/tests/test_historical_retrieval.py::test_historical_features_from_bigquery_sources[True-gcp] 
Time to execute job_from_df.to_df() = '0:00:03.383327'
Time to execute job_from_df.to_df() = '0:00:03.396782'

PASSED [ 70%]
sdk/python/tests/test_historical_retrieval.py::test_historical_features_from_bigquery_sources[True-gcp_custom_offline_config] 
Time to execute job_from_df.to_df() = '0:00:03.640479'
Time to execute job_from_df.to_df() = '0:00:03.284708'

So, 3 seconds? Looks good to me, but curious how much time this took before then?

Copy link
Member

@achals achals left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@feast-ci-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: achals, MattDelac

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@feast-ci-bot feast-ci-bot merged commit 3129dc8 into feast-dev:master Jun 8, 2021
@MattDelac MattDelac deleted the make_test_historical_retrieval_longer branch June 8, 2021 22:13
tsotnet pushed a commit that referenced this pull request Jun 17, 2021
Signed-off-by: Matt Delacour <matt.delacour@shopify.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants