Don't load entire bigquery query results in memory

[tsotnet]

Signed-off-by: Tsotne Tabidze tsotne@tecton.ai

What this PR does / why we need it: In BigQueryOfflineStore.get_historical_features Feast calls entity_df_job.result() which causes the entity data frame to be loaded in memory. We only need to wait for the job to be done and for the schema to be returned, while not changing the existing query (because the results of the query are being used for joining with offline feature data). This PR changes the call to entity_df_job.result(max_results=0), which at the same time:

1. Executes the same query in BigQuery, so the table that gets joined does not change
2. Does not load any of the query rows in memory
3. Still allows the schema to be acquired from the resulting object

I checked all other places where we call .result() in the codebase, but none of them needed fixing due to couple of different reasons (happy to elaborate if anyone is curious).

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

Fix BigQuery entity dataframe SQL query results being completely loaded in memory

[codecov-commenter]

Codecov Report

Merging #1638 (2e66974) into master (0d7e858) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1638      +/-   ##
==========================================
+ Coverage   83.59%   83.60%   +0.01%     
==========================================
  Files          67       67              
  Lines        6017     6027      +10     
==========================================
+ Hits         5030     5039       +9     
- Misses        987      988       +1     

Flag	Coverage Δ
integrationtests	83.52% <100.00%> (+0.01%)	⬆️
unittests	76.43% <0.00%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
sdk/python/feast/infra/offline_stores/bigquery.py	92.48% <100.00%> (ø)
sdk/python/tests/test_e2e_local.py	100.00% <0.00%> (ø)
sdk/python/feast/infra/offline_stores/file.py	96.70% <0.00%> (+0.03%)	⬆️
sdk/python/feast/errors.py	66.66% <0.00%> (+0.87%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d7e858...2e66974. Read the comment docs.

[achals]

/lgtm

[feast-ci-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: achals, tsotnet

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [achals,tsotnet]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment