Add support for Azure cloud storage

[hlky]

Description of changes:

Add support for Azure cloud storage.

Authentication with storage account shared key aka account key or access key

use environment variables AZURE_ACCOUNT_NAME and AZURE_ACCOUNT_ACCESS_KEY

test coverage is minimal, I didn't find a library to mock Azure, I suppose I can add more test coverage if absolutely required, but it does work

>>> import numpy as np
>>> from PIL import Image
>>> from streaming import MDSWriter
>>>
>>> # Local or remote directory in which to store the compressed output files
>>> data_dir = 'azure://test'
>>>
>>> # A dictionary mapping input fields to their data types
>>> columns = {
...     'image': 'jpeg',
...     'class': 'int'
... }
>>>
>>> # Shard compression, if any
>>> compression = 'zstd'
>>>
>>> # Save the samples as shards using MDSWriter
>>> with MDSWriter(out=data_dir, columns=columns, compression=compression) as out:
...     for i in range(10000):
...         sample = {
...             'image': Image.fromarray(np.random.randint(0, 256, (32, 32, 3), np.uint8)),
...             'class': np.random.randint(10),
...         }
...         out.write(sample)
...
>>>
>>> from torch.utils.data import DataLoader
>>> from streaming import StreamingDataset
>>>
>>> # Remote path where full dataset is persistently stored
>>> remote = 'azure://test'
>>>
>>> # Local working dir where dataset is cached during operation
>>> local = './test'
>>>
>>> # Create streaming dataset
>>> dataset = StreamingDataset(local=local, remote=remote, shuffle=True)
>>>
>>> # Let's see what is in sample #1337...
>>> sample = dataset[1337]
>>> img = sample['image']
>>> cls = sample['class']
>>>
>>> # Create PyTorch DataLoader
>>> dataloader = DataLoader(dataset)
>>> sample
{'class': 9, 'image': <PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=32x32 at 0x2DACE9C55D0>}

Issue #, if available:

#254

Merge Checklist:

Put an x without space in the boxes that apply. If you are unsure about any checklist, please don't hesitate to ask. We are here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

• I have read the contributor guidelines
• This is a documentation change or typo fix. If so, skip the rest of this checklist.
• I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the MosaicML team.
• I have updated any necessary documentation, including README and API docs (if appropriate).

Tests

• I ran pre-commit on my change. (check out the pre-commit section of prerequisites)
• I have added tests that prove my fix is effective or that my feature works (if appropriate).
• I ran the tests locally to make sure it pass. (check out testing)
• I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes.

[karan6181]

@hlky Thanks for creating a PR. Can you please re-base your branch since there is a conflicting files ? Thanks!

[karan6181]

Can you please also import R2Uploader and AzureUploader in streaming/base/storage/__init__.py ?

[karan6181]

LGTM. Thank You!