updates retrieve command to read from cache #33
Conversation
Signed-off-by: Bruno Calza <brunoangelicalza@gmail.com>
| return nil | ||
| } | ||
|
|
||
| type coldStore struct{} // nolint |
There was a problem hiding this comment.
unused code, but keeping for future
internal/app/retriever.go
Outdated
| provider VaultsProvider | ||
| } | ||
|
|
||
| func (cs *cacheStore) retrieveStdout(ctx context.Context, cid cid.Cid) error { |
There was a problem hiding this comment.
writing to stdout, we simply pass stdout as the io.Writer
internal/app/retriever.go
Outdated
| return nil | ||
| } | ||
|
|
||
| func (cs *cacheStore) retrieveFile(ctx context.Context, cid cid.Cid, output string, name string) error { |
There was a problem hiding this comment.
writing to file, we must ensure the dir exists, then we open the file, and pass as the io.writer
| ) | ||
|
|
||
| // Retriever is responsible for retrieving file from the network. | ||
| type Retriever struct { |
There was a problem hiding this comment.
new component, that has two modes of retrieval, only the cache is being used right now
cmd/vaults/commands.go
Outdated
| if err != nil { | ||
| return fmt.Errorf("failed to write to stdout: %s", err) | ||
| } | ||
| retriever := app.NewRetriever(vaultsprovider.New(provider)) |
There was a problem hiding this comment.
Did a bit of refactoring and created a new component
| @@ -614,6 +607,15 @@ func newRetrieveCommand() *cli.Command { | |||
| DefaultText: "current directory", | |||
| Destination: &output, | |||
| }, | |||
| &cli.StringFlag{ | |||
There was a problem hiding this comment.
added this flag to make it easier to test on other providers
| "github.com/ipfs/go-cid" | ||
| "github.com/stretchr/testify/require" | ||
| ) | ||
|
|
There was a problem hiding this comment.
one test for stdout and another for file
| @@ -282,3 +282,10 @@ func (bp *vaultsProviderMock) WriteVaultEvent( | |||
| close(bp.uploaderInputs) | |||
| return nil | |||
| } | |||
|
|
|||
| func (bp *vaultsProviderMock) RetrieveEvent( | |||
There was a problem hiding this comment.
the mocked implementation used in tests
pkg/vaultsprovider/provider.go
Outdated
| } | ||
|
|
||
| client := &http.Client{ | ||
| Timeout: 0, |
There was a problem hiding this comment.
Do we want this timeout to be configurable? Maybe we can address that later?
There was a problem hiding this comment.
You mean, to let the client choose the timeout? The problem is which value to put. There's no way to know how long the download will last, that's very dependent on the client's internet connection and file size. but yeah, maybe we could add that extra flexibility if clients want to put an upper bound
There was a problem hiding this comment.
Yes, i meant an upper bound. Maybe something for later.
|
|
||
| type coldStore struct{} // nolint | ||
|
|
||
| func (cs *coldStore) retrieve(ctx context.Context, c cid.Cid, path string) error { // nolint |
|
|
||
| func (cs *cacheStore) retrieveFile(ctx context.Context, cid cid.Cid, output string, name string) error { | ||
| // Write to the provided path or current directory | ||
| if output == "" { |
There was a problem hiding this comment.
Looks good. But curious why not use output == "" for "stdout" instead of hyphen and output == "." for current directory?
There was a problem hiding this comment.
It's because "-" as stdout is a convention commonly used in command-line programs
Signed-off-by: Bruno Calza <brunoangelicalza@gmail.com>
This PR updates the retrieve command to download the file from the cache. I took the opportunity to do some refactoring as well.
Changes:
--cacheflag that defaults totrue. The only way of retrieval right now.