Exercise 1: Working with Azure Storage
Azure Storage is a general purpose storage service for storing data in Azure. It is widely used byapplications and can provide distributed storage services for Big Data technologies. In this exercise, you will create an Azure Storage account and use it to store files.
Provision an Azure Storage Account and a Blob Container
You will use an Azure storage account to store data files. The files will be stored as binary large objects(BLOBs) in a blob store container that is hosted in your storage account.
- In a web browser, navigate to http://portal.azure.com, and if prompted, sign in using the Microsoft account that is associated with your Azure subscription.
- In the Microsoft Azure portal, launch Cloud Shell from the top navigation of the Azure portal.
- Select Bash environment
- Check that the environment drop-down from the left-hand side of shell window says Bash.
- Set your subscription
az account list
az account set --subscription my-subscription-name
- Create a resource group. Every storage account must belong to an Azure resource group. A resource group is a logical container for grouping your Azure services.
az group create \ --name storage-quickstart-resource-group \ --location westus
- Next, create a general-purpose v2 storage account with locally-redundant storage.
az storage account create \ --name storagequickstart \ --resource-group storage-quickstart-resource-group \ --location westus \ --sku Standard_LRS \ --kind StorageV2 --access-tier hot
- The Azure CLI needs your storage account credentials for the rest of the commands
- view keys
az storage account keys list \ --account-name storagequickstart \ --resource-group storage-quickstart-resource-group \ --output table
export AZURE_STORAGE_ACCOUNT="storagequickstart" export AZURE_STORAGE_ACCESS_KEY=$(az storage account keys list \ --account-name storagequickstart \ --resource-group storage-quickstart-resource-group \ --query JMESPath)
- Blobs are always uploaded into a container. You can organize groups of blobs similar to the way you organize your files on your computer in folders.
az storage container create \ --name mystoragecontainer \ --public--access off
- List the blobs in a container
az storage blob list \ --container-name mystoragecontainer \ --output table
After the bigdata blob container has been created.
Sign in to azure portal. From
storage accounts and click the account you have created. Click on
blobs and the container you created. Click Properties and view the URL for the blob container, which should be inthe form https://
In this example, you upload a blob to the container you created in the last step using the az storage blob upload command.
az storage blob upload \ --container-name mystoragecontainer \ --name blobName \ --file ~/path/to/local/file
Use the az storage blob download command to download the blob you uploaded earlier.
az storage blob download \ --container-name mystoragecontainer \ --name blobName \ --file ~/destination/path/for/file