Connecting to Azure Blob Storage: A Guide
Overview
This document provides a comprehensive guide for customers on how to connect their applications and services to Azure Blob Storage. It outlines the specific inputs required to establish a secure connection using an Account Key and how to specify the target container and folder for data transfers. This process applies whether you are using Azure Blob Storage as a data source (reading data) or a data destination (writing data).Prerequisites
Before you begin, please ensure you have the following:- An active Microsoft Azure account.
- An existing Azure Storage Account created within your account.
- The necessary permissions within your Azure account to access the storage account’s security credentials.
Authentication: Using the Connection String
To connect to your Azure Storage Account, we use a Connection String. This string acts as a secure password, containing all the necessary information to authenticate your access. It is the only credential needed to establish the connection.Information Required from You
To create or retrieve this connection string, you will need two key pieces of information:- Storage Account Name: The unique name you assigned to your Azure Storage account.
- Connection String: The access string obtained from the procedure above.
- Filename Prefix: The way you would identify your file. Data on naming convention is given below.
- Account Key: A secret access key that provides administrative rights to the storage account.
How to Find Your Connection String in Azure
Follow these steps to locate the required credentials in your Azure Portal:Log in to the Azure Portal: Open your web browser and navigate to https://portal.azure.com.
Navigate to Storage Account: Use the main search bar at the top of the portal to search for and select your storage account.
Access Security Keys: In the left-hand menu for your storage account, find the section labeled Security + networking and click on Access keys.
Retrieve Credentials:This single string is what you will provide to the application for authentication.
- The Storage account name is clearly displayed at the top of this page.
- You will see two keys, key1 and key2. Click the Show button next to either key.
- For your convenience, Azure provides the complete Connection string. Click the copy icon next to the connection string to copy it to your clipboard.
Destination Configuration
Once authentication is established with the connection string, you must specify the exact location within the storage account for your data. These “Path Inputs” are provided separately from the connection string.Information Required from You
- Container Name: The name of the specific container where data will be read from or written to.
- Folder Name (Optional): The virtual folder path within the container. If no folder is specified, data will be placed in the root of the container.
How to Find Your Path Inputs in Azure
Identify Container Name: You will see a list of all containers within this storage account. Copy the exact name of the container you wish to use.
Identify Folder Name:
- Click on the desired container name to view its contents.
- Azure Blob Storage uses a virtual folder structure. These folders are created by including slashes in the blob names (e.g., a file named processed/invoices/inv_123.pdf creates the virtual folders processed and invoices).
- Determine the folder path you intend to use. For example, input-files or processed/invoices.
Create a Destination for Azure Blob Storage
Perform the following steps to create a Destination for Azure Blob Storage:
Click Azure Blob Storage. A screen appears displaying details about the particular destination towards the left. On the right-hand side of the screen find a list of fields that are required for the integration to be established. Enter the required details as mentioned in the following steps:a. Enter a name for the Destination.b. Enter the Access token that you have obtained from your Azure instance. The steps to get the access token is mentioned in the Prerequisites section of this document.
c. Enter the Container, Connection String, Prefix name and Folder of your Azure instance. The steps to get the Container is mentioned in the Prerequisites section of this document.d. Under Filename Prefix, enter a text value without special characters, such as underscores (_), spaces, or symbols (
Example File Name:
#, @, $, %, &, \*). Note that the file name follows the format:
${filenamePrefix}part-${sequence}_${destinationAudienceId}_${timestamp}.csv, where the timestamp is in yyyy-MM-dd format and destinationAudienceId is the ID associated with your audience (segment) in Zeotap CDP, which is generated by the Zeotap CDP after the audience (segment) is linked to a Destination.| Variable | Description |
|---|---|
filenamePrefix | The prefix for the file name, typically indicating the context or purpose of the file (e.g., the segment or export type). |
part-${sequence} | A sequential number or identifier indicating the part or chunk number if the data is split across multiple files. |
${destinationAudienceId} | The unique ID assigned by Zeotap CDP to the audience (segment) after it is linked to a Destination. |
${timestamp} | The date when the file is generated, formatted as yyyy-MM-dd (e.g., 2024-08-06). |
.csv | The file extension, indicating that the file is in CSV (Comma-Separated Values) format. |
exportpart-01_894548842_2024-08-06.csve. Review all the values entered above and then click Next to proceed.In the new screen that appears, choose the appropriate action and mapping as explained below. Under Choose your Action, choose Send Complete CSV file to Blob Storage as the action.You can map the following fieldsi. Identifiersii. Any other custom field using the +Add Mapping button