Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generated empty messages without proto.Clone. #69

Closed
wants to merge 1 commit into from

Conversation

chhsia0
Copy link
Contributor

@chhsia0 chhsia0 commented Jan 3, 2020

This PR partially fixes #27.

The resource.Quantity proto message is generated from a go struct through the go-to-protobuf k8s tool. This creates two problems:

  • The Quantity go struct has unexported fields, which is not cloneable through reflection, but skycfg uses proto.Clone to create empty messages. This problem is fixed in this PR by creating empty messages through reflect.New instead.
  • The Quantity go struct doesn’t have any protobuf tag, but skycfg uses tags to extract field names. As a result, skycfg doesn’t not recognize resource.Quantity(string = "1000m"). To resolve this, you'll need to replace k8s.io/apimachinery in go.mod with my fork, which adds protobuf tags to the Quantity go struct:
replace k8s.io/apimachinery => github.com/chhsia0/apimachinery <k8s_version>-quantity-tag

where k8s_version can be 1.14, 1.15, 1.16 or 1.17. I'll post a PR to upstream this change.

@jmillikin-stripe
Copy link
Contributor

Please file a bug against Kubernetes, asking them to use valid Protobuf implementations instead of doing their own half-baked thing. Skycfg is designed to operate on Protobuf messages, not arbitrary Go structs, and we don't have plans to support Go structs that falsely claim to be Protobuf messages.

Note that the Kubernetes code may break anyway once golang/protobuf#364 lands in upstream go-protobuf. And it will definitely not work once support for dynamic messages is implemented in go-protobuf, since at that point a message type might not be backed by any concrete Go type at all.

You may be able to work around this by updating your build workflow to generate Go source from the Kubernetes .proto files using the standard Protobuf tooling. This is what we (Stripe) do, using Bazel as our build tool. Other build tools -- Make, Rake, CMake, basically anything that can run commands -- should also be usable in this role.

@dilyevsky
Copy link
Contributor

dilyevsky commented Jan 4, 2020

fwiw, we wrapped this thing with a special helper in Isopod (https://github.com/cruise-automation/isopod/blob/19a7f547c270318529df74a59f47bc5684837ec8/pkg/kube/util.go#L29) and never had any issues.

Examples usage:

... = corev1.ResourceRequirements(requests = {"cpu": kube.resource_quantity("1000m")})

@jbarrick-mesosphere
Copy link

Thank you @dilyevsky !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

whats the correct way to convert non protobuf fields ?
4 participants