Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add SHOW TOPICS EXTENDED #3183

Merged

Conversation

cpettitt-confluent
Copy link
Contributor

Fixes #1268

BREAKING CHANGE: "SHOW TOPICS" no longer includes the "Consumers" and
"ConsumerGroups" columns. You can use "SHOW TOPICS EXTENDED" to get the
output previous emitted from "SHOW TOPICS". See below for examples.

This change splits "SHOW TOPICS" into two commands:

  1. "SHOW TOPICS EXTENDED", which shows what was previously shown by
    "SHOW TOPICS". Sample output:
    ksql> show topics extended;

     Kafka Topic                                                                                   | Partitions | Partition Replicas | Consumers | ConsumerGroups
    --------------------------------------------------------------------------------------------------------------------------------------------------------------
     _confluent-command                                                                            | 1          | 1                  | 1         | 1
     _confluent-controlcenter-5-3-0-1-actual-group-consumption-rekey                               | 1          | 1                  | 1         | 1
  1. "SHOW TOPICS", which now no longer queries consumer groups and their
    active consumers. Sample output:
    ksql> show topics;

     Kafka Topic                                                                                   | Partitions | Partition Replicas
    ---------------------------------------------------------------------------------------------------------------------------------
     _confluent-command                                                                            | 1          | 1
     _confluent-controlcenter-5-3-0-1-actual-group-consumption-rekey                               | 1          | 1

This changeset primarily involved renaming KafkaTopicXXX classes to
KafkaTopicXXXExtended and then trimming out the consumer group info from
the original KafkaTopicXXX classes.

@ghost
Copy link

ghost commented Aug 6, 2019

It looks like @cpettitt-confluent hasn't signed our Contributor License Agreement, yet.

The purpose of a CLA is to ensure that the guardian of a project's outputs has the necessary ownership or grants of rights over all contributions to allow them to distribute under the chosen licence.
Wikipedia

You can read and sign our full Contributor License Agreement here.

Once you've signed reply with [clabot:check] to prove it.

Appreciation of efforts,

clabot

Copy link
Contributor

@agavra agavra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @cpettitt-confluent! the logic LGTM - the only comment is about code duplication which I think we can reduce and a few nits inline


@JsonIgnoreProperties(ignoreUnknown = true)
@JsonSubTypes({})
public class KafkaTopicInfoExtended {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thoughts on adding a boolean for extended to the original entity? That way we can avoid essentially duplicating a lot of logic in KafkaTopicInfo, KafkaTopicListExtended and KafkaTopicInfoTableBuilder

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I definitely understand the urge to collapse them. Here are a few counter arguments, none of which I'm very attached to:

  1. It allows us to let the two structures vary independently.
  2. It somewhat follows a convention I saw with SourceDescription and SourceInfo::Table (maybe for reason 1)?
  3. It's very clear what is in the entity, which potentially saves us branching and special case code, e.g. introducing Optionals for the "simple" case.

We could, for the moment, compose KafkaTopicInfo in KafkaTopicInfoExtended and re-export the common fields name, replicateInfo, but that doesn't buy us too much and I don't think it gets you the level of unification you're looking for.

Ultimately, I'm happy to follow team convention. If unification looks more like the rest of the code base let me know and I will make it happen.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Funny that you mention SourceDescription - I looked at that before suggesting this. You can see it has a extended boolean that it leverages in Console to determine how it prints the response.

Either way, I'm not attached either to having them be a single entity. Really the important part to me is that the duplicated business logic is shared (e.g. filterKsqlInternalTopics and getTopicReplicaInfo)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @agavra! Totally agree on the business logic. I'll extract those where they're duplicated and follow up.

@agavra agavra requested a review from a team August 6, 2019 23:31
@cpettitt-confluent
Copy link
Contributor Author

Thanks @agavra for the feedback and especially the guidance re. team conventions. I'd love to close on the unification question, if you or anyone else on the team has guidance re. handling similar entities. The rest is straight forward and will be fixed in the next update.

@cpettitt-confluent
Copy link
Contributor Author

@agavra Latest updates address your comments re. copyright date, removal of EasyMock, and usage of comments in test (Given, When, Then).

I'm waiting on the conclusion re. unification to make changes there.

Fixes confluentinc#1268

BREAKING CHANGE: "SHOW TOPICS" no longer includes the "Consumers" and
"ConsumerGroups" columns. You can use "SHOW TOPICS EXTENDED" to get the
output previous emitted from "SHOW TOPICS". See below for examples.

This change splits "SHOW TOPICS" into two commands:

1. "SHOW TOPICS EXTENDED", which shows what was previously shown by
"SHOW TOPICS". Sample output:

```
    ksql> show topics extended;

     Kafka Topic                                                                                   | Partitions | Partition Replicas | Consumers | ConsumerGroups
    --------------------------------------------------------------------------------------------------------------------------------------------------------------
     _confluent-command                                                                            | 1          | 1                  | 1         | 1
     _confluent-controlcenter-5-3-0-1-actual-group-consumption-rekey                               | 1          | 1                  | 1         | 1
```

2. "SHOW TOPICS", which now no longer queries consumer groups and their
active consumers. Sample output:

```
    ksql> show topics;

     Kafka Topic                                                                                   | Partitions | Partition Replicas
    ---------------------------------------------------------------------------------------------------------------------------------
     _confluent-command                                                                            | 1          | 1
     _confluent-controlcenter-5-3-0-1-actual-group-consumption-rekey                               | 1          | 1
```

This changeset primarily involved renaming KafkaTopicXXX classes to
KafkaTopicXXXExtended and then trimming out the consumer group info from
the original KafkaTopicXXX classes.
@cpettitt-confluent
Copy link
Contributor Author

@agavra, I'm pretty happy with how this came out. Let me know what you think.

This change moves all of the logic out of the entities and into the controller (ListTopicsExecutor), which was not the case pre-PR. This allowed me to centralize the filtering logic. I collapsed table building too, which buys us a little de-dup, but not as much. I also removed the non-serde testing in KafkaTopicsListXXXTest since KafkaTopicsListXXX are now just bare entities. We still get testing in ListTopicsExecutorTest.

Copy link
Contributor

@agavra agavra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it way more in the Executor than in the Entity - it makes way more sense! LGTM.

@agavra agavra requested a review from a team August 7, 2019 21:16
@cpettitt-confluent cpettitt-confluent merged commit dd3eb5f into confluentinc:master Aug 7, 2019
@cpettitt-confluent cpettitt-confluent deleted the feature-1268 branch August 7, 2019 21:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add SHOW TOPICS EXTENDED
2 participants