Support reproducible builds

[Monnoroch]

rules_python allows using a requirements.txt file with pinned down versions of dependencies. However, this is not good enough, since the actual archive downloaded can be different each time, which is really bad for Bazel's aggressive caching strategy and for cross-machine reproducibility. Plus, this it totally in spirit of Bazel. I fully realize that this feature will require explicitly listing transitive dependencies but it is totally acceptable when maximum reproducibility is required. The way I would propose to implement it would be a generator that given an input requirements.txt file outputs a list of targets to be added to the WORKSPACE file. These rules should also include sha 256 hashes of compiled wheels or other artifacts. Another important thing to remember is cross-compilation: some Python libraries (such as numpy) include native code and require to be built for a target system. The above generator should support common patterns and be extensible enough for developers to handle tricky cases. Right now, these rules download different files on Windows, so with a linux-only CI environment I experience issues when people that use Windows try to build code.

Related: #140.

Somewhat related: #154.

[alexeagle]

rules_python 0.1.0 has been released which upstreams the rules_python_external repo. Please switch from pip_import to pip_install which doesn't have this issue.

[Monnoroch]

Could you please clarify how exactly is pip_install reproducible? I see that it's still using requirements.txt which means neither we specify sha256 hash for the downloaded archives nor we pin transitive dependencies. From the looks of it, nothing has really changed as far as hermeticity goes, I will still occasionally get different archives downloaded after bazel clean --expunge.

[alexeagle]

I closed this based on the README from rules_python_external.

I agree though, you need an extra bit of pip-compile or equivalent to pin and lock dependencies. I'll send a PR to add that rule.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days.
Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!