On the Wren Solutions project, there was need to sync a large data set from a device and merge data from it into an existing data set in Microsoft Azure. To accomplish this we decided to use the following workflow:
- Upload the file using Azure IoT Hub
- Trigger a function on the Azure Blob creation
Upload the file using Azure IoT Hub
If you haven’t, first you have to create an Azure IoT Hub.
Associate an Azure Storage account to IoT Hub
When you associate an Azure Storage account with an IoT hub, the IoT hub generates a SAS URI. A device can use this SAS URI to securely upload a file to a blob container. The IoT Hub service and the device SDKs coordinate the process that generates the SAS URI and makes it available to a device to use to upload a file.
Make sure that a blob container is associated with your IoT hub and that file notifications are enabled.
To use the file upload functionality in IoT Hub, you must first associate an Azure Storage account with your hub. Select File upload to display a list of file upload properties for the IoT hub that is being modified.
Storage container: Use the Azure portal to select a blob container in an Azure Storage account in your current Azure subscription to associate with your IoT Hub. If necessary, you can create an Azure Storage account on the Storage accounts blade and blob container on the Containersblade. IoT Hub automatically generates SAS URIs with write permissions to this blob container for devices to use when they upload files.
Use Azure IoT SDK to upload blob
Use the Azure IoT Hub C# SDK to upload the file. Below is a Gist of a code sample showing how to upload using the SDK. The code showcases how to utilize UploadToBlobAsync method on the Device Client. To use the sample replace the DeviceConnectionString and the FilePath variable.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| // Copyright (c) Microsoft. All rights reserved. | |
| // Licensed under the MIT license. See LICENSE file in the project root for full license information. | |
| // From the microsoft samples here https://github.com/Azure/azure-iot-sdk-csharp/blob/master/device/samples/DeviceClientFileUploadSample/Program.cs | |
| using System; | |
| using System.IO; | |
| using System.Threading.Tasks; | |
| namespace Microsoft.Azure.Devices.Client.Samples | |
| { | |
| class Program | |
| { | |
| // String containing Hostname, Device Id & Device Key in one of the following formats: | |
| // "HostName=<iothub_host_name>;DeviceId=<device_id>;SharedAccessKey=<device_key>" | |
| // "HostName=<iothub_host_name>;CredentialType=SharedAccessSignature;DeviceId=<device_id>;SharedAccessSignature=SharedAccessSignature sr=<iot_host>/devices/<device_id>&sig=<token>&se=<expiry_time>"; | |
| private const string DeviceConnectionString = "<replace>"; | |
| private const string FilePath = "<replace>"; | |
| static void Main(string[] args) | |
| { | |
| try | |
| { | |
| SendToBlobSample().Wait(); | |
| } | |
| catch (Exception ex) | |
| { | |
| Console.WriteLine("{0}\n", ex.Message); | |
| if (ex.InnerException != null) | |
| { | |
| Console.WriteLine(ex.InnerException.Message + "\n"); | |
| } | |
| } | |
| } | |
| static async Task SendToBlobSample() | |
| { | |
| var deviceClient = DeviceClient.CreateFromConnectionString(DeviceConnectionString, TransportType.Http1); | |
| var fileStreamSource = new FileStream(FilePath, FileMode.Open); | |
| var fileName = Path.GetFileName(fileStreamSource.Name); | |
| Console.WriteLine("Uploading File: {0}", fileName); | |
| var watch = System.Diagnostics.Stopwatch.StartNew(); | |
| await deviceClient.UploadToBlobAsync(fileName, fileStreamSource); | |
| watch.Stop(); | |
| Console.WriteLine("Time to upload file: {0}ms\n", watch.ElapsedMilliseconds); | |
| } | |
| } | |
| } |
Trigger a function on the Azure Blob creation
A Blob storage trigger starts an Azure Function when a new or updated blob is detected. The blob contents are provided as input to the function. Setup the blob trigger to use the container we linked to the Azure IoT Hub previously. First lets configure and manage your function apps in the Azure portal.
To begin, go to the Azure portal and sign in to your Azure account. In the search bar at the top of the portal, type the name of your function app and select it from the list. After selecting your function app, you see the following page:
Go to the Platform Features tab by clicking the tab of the same name.
Function apps run in, and are maintained, by the Azure App Service platform. As such, your function apps have access to most of the features of Azure’s core web hosting platform. The Platform features tab is where you access the many features of the App Service platform that you can use in your function apps.
Add a connection string from the blob storage account as an app setting. For the sack of this demo lets name it MyStorageAccountAppSetting. Reference that in your JSON for you Blob Trigger. Then use that blob name as a reference to that blob in your function.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "disabled": false, | |
| "bindings": [ | |
| { | |
| "name": "myBlob", | |
| "type": "blobTrigger", | |
| "direction": "in", | |
| "path": "samples-workitems", | |
| "connection":"MyStorageAccountAppSetting" | |
| } | |
| ] | |
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| public static void Run(Stream myBlob, TraceWriter log) | |
| { | |
| log.Info($"C# Blob trigger function Processed blob\n Name:{name} \n Size: {myBlob.Length} Bytes"); | |
| } |
