Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add connect templates and simplify JDBC source (MINOR) #3231

Merged
merged 2 commits into from
Aug 19, 2019

Conversation

agavra
Copy link
Contributor

@agavra agavra commented Aug 19, 2019

Description

Adds some nice defaults and shorthand for creating a JDBC source connector.

Before:

CREATE SOURCE CONNECTOR `jdbc-connector3` 
WITH("connector.class"='io.confluent.connect.jdbc.JdbcSourceConnector',
"tasks.max"='1',
"connection.url"='jdbc:postgresql://localhost:5432/almog.gavra',
"mode"='bulk'
,"topic.prefix"='jdbc3-',
"transforms"='createKey,extractString',
"transforms.createKey.type"= 'org.apache.kafka.connect.transforms.ValueToKey', 
"transforms.createKey.fields"='username',
"transforms.extractString.type"='org.apache.kafka.connect.transforms.ExtractField$Key',
"transforms.extractString.field"='username', 
"key.converter"='org.apache.kafka.connect.storage.StringConverter');

After:

CREATE SOURCE CONNECTOR `jdbc-connector` 
WITH("connector.class"='io.confluent.connect.jdbc.JdbcSourceConnector',
"connection.url"='jdbc:postgresql://localhost:5432/almog.gavra',
"mode"='bulk',
"topic.prefix"='jdbc-', 
"key"='username');

TODO

  • Documentation will be part of end-to-end Connector documentation

Testing done

  • Unit testing
  • End to end local testing

Reviewer checklist

  • Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
  • Ensure relevant issues are linked (description should include text like "Fixes #")

@agavra agavra requested a review from a team as a code owner August 19, 2019 21:23
Copy link
Contributor

@vcrfxia vcrfxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @agavra -- LGTM!

ConnectTemplate.resolve(
Maps.transformValues(
createConnector.getConfig(),
l -> l != null ? l.getValue().toString() : null)));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would this ever be null?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

never... it's just that intelliJ complains when I don't check for it because transformValues has an annotation (@NullableDecl)

@vcrfxia vcrfxia requested a review from a team August 19, 2019 21:53
import java.util.stream.Collectors;
import org.apache.kafka.connect.runtime.rest.entities.ConnectorInfo;

public enum Connectors implements SupportedConnector {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this implement SupportConnector?

@agavra agavra merged commit ba0fb99 into confluentinc:master Aug 19, 2019
@agavra agavra deleted the connect_jdbc branch August 19, 2019 23:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants