Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Unix Timestamp value type #1520

Merged

Conversation

MattDelac
Copy link
Collaborator

@MattDelac MattDelac commented Apr 28, 2021

What this PR does / why we need it:
As discussed in Slack, so far, we don't want to support datetime dtypes in general. Therefore, so far we would need to convert any datetime data types into a unix timestamp and store it as an INT in feast.

While converting in unix timestamp is not an issue, it makes it hard to differentiate the features that are int vs timestamp when looking at a FeatureView

Thus this PR aims to introduce a new datatype "Unix Timestamp" that beyond the scene would be an INT. For the end user, this will make it easier understand a FeatureView.
It also add a potential possibility to do things differently with Unix Timestamp features at retrieval time.

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

Add UnixTimestamp valueType that is an Int64 under the hood

@feast-ci-bot
Copy link
Collaborator

Hi @MattDelac. Thanks for your PR.

I'm waiting for a feast-dev member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@woop
Copy link
Member

woop commented Apr 28, 2021

/ok-to-test

@MattDelac MattDelac force-pushed the add_unix_timestamp_value_type branch 4 times, most recently from 7568841 to 0bbe997 Compare April 29, 2021 01:03
@MattDelac MattDelac marked this pull request as ready for review April 29, 2021 15:13
@MattDelac MattDelac requested review from jklegar, tsotnet, woop and a team as code owners April 29, 2021 15:13
@MattDelac MattDelac changed the title WIP: Add Unix Timestamp value type Add Unix Timestamp value type Apr 29, 2021
@woop
Copy link
Member

woop commented Apr 29, 2021

/kind feature

@feast-ci-bot feast-ci-bot added kind/feature New feature or request and removed needs-kind labels Apr 29, 2021
@woop woop self-assigned this Apr 29, 2021
@woop
Copy link
Member

woop commented Apr 29, 2021

Hey @MattDelac, thanks for the contribution!

Overall it looks good to me. One thing that is a glaring omissions (not with your PR but with our code) is that we're not testing all the different types. Regardless, we can add that test a bit later. I've added a note to add a test here.

I think the only thing missing here is an update to our documentation. You may need to update the specs to document the new type: https://github.com/feast-dev/feast/tree/master/docs/specs

@MattDelac
Copy link
Collaborator Author

One thing that is a glaring omissions (not with your PR but with our code) is that we're not testing all the different types.

Yes that was my surprise that none of the tests had to be changed

I've added a note to add a test here.

Sounds good. I let you fix it later 👌

I think the only thing missing here is an update to our documentation

Good catch ! Will look into the doc

@MattDelac MattDelac force-pushed the add_unix_timestamp_value_type branch from a89bff6 to c98dd86 Compare April 29, 2021 18:34
@MattDelac
Copy link
Collaborator Author

I think the only thing missing here is an update to our documentation. You may need to update the specs to document the new type: https://github.com/feast-dev/feast/tree/master/docs/specs

Done. The docs are now up to date. Let me know if I missed another one.

@woop
Copy link
Member

woop commented Apr 30, 2021

/lgtm

@woop
Copy link
Member

woop commented May 4, 2021

/lgtm

@woop woop closed this May 4, 2021
@woop woop reopened this May 4, 2021
@feast-ci-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: MattDelac, woop

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Matt Delacour added 3 commits May 4, 2021 18:56
Signed-off-by: Matt Delacour <matt.delacour@shopify.com>
Signed-off-by: Matt Delacour <matt.delacour@shopify.com>
Signed-off-by: Matt Delacour <matt.delacour@shopify.com>
@MattDelac MattDelac force-pushed the add_unix_timestamp_value_type branch from c4c49c0 to 48f31d0 Compare May 4, 2021 22:57
@MattDelac
Copy link
Collaborator Author

/lgtm

@feast-ci-bot
Copy link
Collaborator

@MattDelac: you cannot LGTM your own PR.

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@woop
Copy link
Member

woop commented May 5, 2021

/lgtm

@feast-ci-bot feast-ci-bot merged commit c674bea into feast-dev:master May 5, 2021
@MattDelac MattDelac deleted the add_unix_timestamp_value_type branch May 5, 2021 12:49
jklegar pushed a commit to jklegar/feast that referenced this pull request May 12, 2021
* Add Unix Timestamp value type

Signed-off-by: Matt Delacour <matt.delacour@shopify.com>

* Update docs

Signed-off-by: Matt Delacour <matt.delacour@shopify.com>

* Handle Scalar type

Signed-off-by: Matt Delacour <matt.delacour@shopify.com>
jklegar pushed a commit that referenced this pull request May 12, 2021
* Add Unix Timestamp value type

Signed-off-by: Matt Delacour <matt.delacour@shopify.com>

* Update docs

Signed-off-by: Matt Delacour <matt.delacour@shopify.com>

* Handle Scalar type

Signed-off-by: Matt Delacour <matt.delacour@shopify.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants