Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] Pulsar Producer & Consumer #921

Merged
merged 11 commits into from
Sep 20, 2023
Merged

[ENH] Pulsar Producer & Consumer #921

merged 11 commits into from
Sep 20, 2023

Conversation

HammadB
Copy link
Collaborator

@HammadB HammadB commented Aug 1, 2023

Description of changes

Summarize the changes made by this PR.

  • New functionality
    • Adds a basic pulsar producer, consumer and associated tests. As well as a docker compose for the distributed version of chroma.

Test plan

We added bin/cluster-test.sh, which starts pulsar and allows test_producer_consumer to run the pulsar fixture.

Documentation Changes

None required.

@github-actions
Copy link

github-actions bot commented Aug 1, 2023

Reviewer Checklist

Please leverage this checklist to ensure your code review is thorough before approving

Testing, Bugs, Errors, Logs, Documentation

  • Can you think of any use case in which the code does not behave as intended? Have they been tested?
  • Can you think of any inputs or external events that could break the code? Is user input validated and safe? Have they been tested?
  • If appropriate, are there adequate property based tests?
  • If appropriate, are there adequate unit tests?
  • Should any logging, debugging, tracing information be added or removed?
  • Are error messages user-friendly?
  • Have all documentation changes needed been made?
  • Have all non-obvious changes been commented?

System Compatibility

  • Are there any potential impacts on other parts of the system or backward compatibility?
  • Does this change intersect with any items on our roadmap, and if so, is there a plan for fitting them together?

Quality

  • Is this code of a unexpectedly high quality (Readbility, Modularity, Intuitiveness)

@@ -0,0 +1,39 @@
syntax = "proto3";
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: add a readme on how to generate protos

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: mypy ignore protos

@HammadB HammadB changed the title Pulsar Producer [WIP/Proto] Pulsar Producer Aug 5, 2023
## Description of changes

*Summarize the changes made by this PR.*
 - New functionality
	 - Adds a pulsar consumer 
	 - Add pulsar admin api
- Hardcoded to default tenant/namespace. Need to support this in
coordinator.
	 
## Test plan
Will add fixtures for this config

## Documentation Changes
None
@HammadB HammadB changed the title [WIP/Proto] Pulsar Producer [ENH] Pulsar Producer & Consumer Sep 20, 2023
@HammadB HammadB requested a review from atroyn September 20, 2023 01:02
Copy link
Contributor

@atroyn atroyn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, tests also pass for me locally.

@HammadB HammadB enabled auto-merge (squash) September 20, 2023 05:21
@HammadB HammadB merged commit 8968222 into main Sep 20, 2023
84 checks passed
tazarov pushed a commit to amikos-tech/chroma-core that referenced this pull request Sep 21, 2023
## Description of changes

*Summarize the changes made by this PR.*
 - New functionality
- Adds a basic pulsar producer, consumer and associated tests. As well
as a docker compose for the distributed version of chroma.

## Test plan
We added bin/cluster-test.sh, which starts pulsar and allows
test_producer_consumer to run the pulsar fixture.

## Documentation Changes
None required.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants