Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Add online_read_async for dynamodb #4244

Merged
merged 16 commits into from
Jun 5, 2024
+790 −156
Merged

feat: Add online_read_async for dynamodb #4244

Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
bb0a4fb
add aiobotocore and online_read_async to dynamo read
May 31, 2024
6b178bb
format
May 31, 2024
a19d46d
update test to run on dynamo
May 31, 2024
d7a982f
duck type the table
May 31, 2024
3db88cb
Merge pull request #5 from robhowley/rh-async-dynamo
robhowley May 31, 2024
ee0b634
Merge branch 'feast-dev:master' into master
robhowley May 31, 2024
0217149
update requirements files
May 31, 2024
eeb3929
fix: gotta await
May 31, 2024
c4ba283
just pass full method, not client
May 31, 2024
8c164c3
chore: allow for some duplication to make await cleaner/more obvious
May 31, 2024
a7488e3
fix: client vs resource payloads
May 31, 2024
a1d1ada
fix resource vs client response handling
May 31, 2024
6bd29f7
deserialize client response so that it looks like the table resource …
May 31, 2024
159259b
temp disable due to flakiness in tests
Jun 2, 2024
fc9e603
run linter
Jun 2, 2024
42e98d2
relax to >2,<3
Jun 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
format 
Signed-off-by: robhowley <rhowley@seatgeek.com>
  • Loading branch information
robhowley committed May 31, 2024
commit 6b178bbbf2ad5942583268a7098446ddcabc6753
8 changes: 5 additions & 3 deletions sdk/python/feast/infra/online_stores/dynamodb.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,9 @@

try:
import boto3
from aiobotocore import session
from botocore.config import Config
from botocore.exceptions import ClientError
from aiobotocore import session
except ImportError as e:
from feast.errors import FeastExtrasDependencyImportError

Expand Down Expand Up @@ -208,7 +208,9 @@ def online_write_batch(
)
self._write_batch_non_duplicates(table_instance, data, progress, config)

def _read_batches(self, online_config, entity_ids, table_name, batch_get_item) -> List[Tuple[Optional[datetime], Optional[Dict[str, ValueProto]]]]:
def _read_batches(
self, online_config, entity_ids, table_name, batch_get_item
) -> List[Tuple[Optional[datetime], Optional[Dict[str, ValueProto]]]]:
result: List[Tuple[Optional[datetime], Optional[Dict[str, ValueProto]]]] = []

batch_size = online_config.batch_size
Expand Down Expand Up @@ -335,7 +337,7 @@ async def online_read_async(
online_config,
entity_ids,
_get_table_name(online_config, config, table),
client.batch_get_item
client.batch_get_item,
)

def _get_aioboto_session(self):
Expand Down