Speaking at DotNetSouth.Tech

I look forward to speaking on AI on the Edge at DotNetSouth.Tech. This year is the conference’s first year so check it out.

AI on the Edge

The next evolution in cloud computing is a smarter application not in the cloud. As the cloud has continued to evolve, the applications that utilize it have had more and more capabilities of the cloud. This presentation will show how to push logic and machine learning from the cloud to an edge application. Afterward, creating edge applications which utilize the intelligence of the cloud should become effortless.


Speaking at Orlando Code Camp

I am happy to announce I will speaking at the Orlando Code Camp again this year. I will be presenting AI on the Edge, a look into Microsoft’s Azure IoT Edge.


AI on the Edge


The next evolution in cloud computing is a smarter application not in the cloud. As the cloud has continued to evolve, the applications that utilize it have had more and more capabilities of the cloud. This presentation will show how to push logic and machine learning from the cloud to an edge application. Afterward, creating edge applications which utilize the intelligence of the cloud should become effortless.

Hacking Izon Cameras and using Azure IoT Edge

After Izon announced that they were closing down their services (leaving the cameras I already owned useless), I decided to turn them into something useful using Azure. First let me list some resources:

Use the Will it hack link to get access to the mobileye website and verify that the Izon device is still streaming and still working. If it is working, you are already done with edits to the device unless you would like to change the passwords (which you should).

Our goals are as follows:

  • Process the video feed from the Izon camera (we will cheat this early on and only use the image feed)
    • Check for motion
    • Check for faces
    • Check if faces are white listed
    • Check for my dog
  • Process the audio feed
    • Check for any noise
    • Check for non human noises
    • Check for dog barks
    • Check for my and my wife’s voice

These are all stretch goals that will be referred back to as the project moves forward.

Create the Azure IoT Edge module

For the first module, we will use the C Module base image. We are looking for two things from this module:

  • Download the picture feed and pass it to the Edge Hub
  • Download the audio feed and pass it to the Edge Hub

If you don’t know where to get started with the C module of the Azure IoT Edge platform, there is helpful information on the Azure Documentation page. Once the C module is created and ready for editing, we are going to connect to the image feed from the devices. To make this simple, both feeds will be retrieved using HTTP. For the video feed, its simple enough to grab images from the Izon camera existing camera feed.

Now one thing we need, is to be able to connect to each camera within the local network shared with the Edge. Since we would like to be able to add and remove cameras, we will use the device twin to update and manage the list of IP address. The code for updating the list is as follows:

#include <stdio.h>
#include <stdlib.h>
#include "iothub_module_client_ll.h"
#include "iothub_client_options.h"
#include "iothub_message.h"
#include "azure_c_shared_utility/threadapi.h"
#include "azure_c_shared_utility/crt_abstractions.h"
#include "azure_c_shared_utility/platform.h"
#include "azure_c_shared_utility/shared_util_options.h"
#include "iothubtransportmqtt.h"
#include "iothub.h"
#include "time.h"
#include "parson.h"
typedef struct IP_ADDRESS_NODE
const char * address;
struct IP_ADDRESS_NODE * next;
} ip_address_node;
ip_address_node * add_address(const char * address,ip_address_node * previous)
ip_address_node * new_node = (ip_address_node *)malloc(sizeof(ip_address_node));
new_node->address = address;
new_node->next = NULL;
if (previous == NULL)
return new_node;
previous->next = new_node;
return previous;
void delete_address(ip_address_node * current)
if (current == NULL)
ip_address_node * root_node = NULL;
ip_address_node * add_address_to_root(const char * address)
if (root_node = NULL)
root_node = add_address(address,root_node);
ip_address_node * current = root_node;
while(current->next != NULL)
current = current->next;
static void moduleTwinCallback(DEVICE_TWIN_UPDATE_STATE update_state, const unsigned char* payLoad, size_t size, void* userContextCallback)
printf("\r\nTwin callback called with (state=%s, size=%zu):\r\n%s\r\n",
ENUM_TO_STRING(DEVICE_TWIN_UPDATE_STATE, update_state), size, payLoad);
JSON_Value *root_value = json_parse_string(payLoad);
JSON_Object *root_object = json_value_get_object(root_value);
JSON_Array * ipaddresses = json_object_dotget_array(root_object, "desired.CameraAddresses");
if (ipaddresses != NULL) {
for (int i = 0; i < json_array_get_count(ipaddresses); i++) {
ipaddresses = json_object_get_array(root_object, "CameraAddresses");
if (ipaddresses != NULL) {
for (int i = 0; i < json_array_get_count(ipaddresses); i++) {

view raw


hosted with ❤ by GitHub

With that code in place, the list of IP addresses can be updated from the Azure UI and the Azure Service SDKs.

Downloading from the Image feed

The Izon cameras make downloading the image feed trivial. There is an existing endpoint where you can grab the latest image directly from the camera’s web server. The latest image is always at /cgi-bin/img-d1.cgi. (NOTE: if you are checking this image from a browser, be sure to have some cache busting!). To download this image into our module, we will use the Curl library for it’s easy HTTP implementation. To add Curl to our Edge module, we will add the following lines to the Dockerfile.amd64.debug:

FROM ubuntu:xenial AS base
RUN apt-get update && \
apt-get install -y –no-install-recommends software-properties-common gdb && \
add-apt-repository -y ppa:aziotsdklinux/ppa-azureiot && \
apt-get update && \
apt-get install -y azure-iot-sdk-c-dev && \
rm -rf /var/lib/apt/lists/*
FROM base AS build-env
RUN apt-get update && \
apt-get install -y –no-install-recommends cmake gcc g++ make libcurl4-openssl-dev && \
rm -rf /var/lib/apt/lists/*
COPY . ./
RUN make
FROM base
COPY –from=build-env /app ./
CMD ["./main"]

FROM ubuntu:xenial AS base
RUN apt-get update && \
apt-get install -y –no-install-recommends software-properties-common gdb && \
add-apt-repository -y ppa:aziotsdklinux/ppa-azureiot && \
apt-get update && \
apt-get install -y azure-iot-sdk-c-dev && \
rm -rf /var/lib/apt/lists/*
FROM base AS build-env
RUN apt-get update && \
apt-get install -y –no-install-recommends cmake gcc g++ make && \
rm -rf /var/lib/apt/lists/*
COPY . ./
RUN make
FROM base
COPY –from=build-env /app ./
CMD ["./main"]

With curl now added to the image, it can be utilized in code by adding it to the method invoked in our main loop. The code will download the file for each entry in the IP address list. Once the image is downloaded, it will send it as a message to the Edge Hub and add the IP address of the camera to the message header. Here is that code:

struct MemoryStruct {
char *memory;
size_t size;
static size_t
WriteMemoryCallback(void *contents, size_t size, size_t nmemb, void *userp)
size_t realsize = size * nmemb;
struct MemoryStruct *mem = (struct MemoryStruct *)userp;
char *ptr = realloc(mem->memory, mem->size + realsize + 1);
if(ptr == NULL) {
/* out of memory! */
printf("not enough memory (realloc returned NULL)\n");
return 0;
mem->memory = ptr;
memcpy(&(mem->memory[mem->size]), contents, realsize);
mem->size += realsize;
mem->memory[mem->size] = 0;
return realsize;
struct MemoryStruct download_file(const char * address)
struct MemoryStruct chunk;
chunk.memory = malloc(1); /* will be grown as needed by the realloc above */
chunk.size = 0; /* no data at this point */
/* init the curl session */
CURL * curl_handle = curl_easy_init();
/* specify URL to get */
curl_easy_setopt(curl_handle, CURLOPT_URL, address);
/* send all data to this function */
curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);
/* we pass our 'chunk' struct to the callback function */
curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, (void *)&chunk);
/* some servers don't like requests that are made without a user-agent
field, so we provide one */
curl_easy_setopt(curl_handle, CURLOPT_USERAGENT, "libcurl-agent/1.0");
/* get it! */
CURLcode res = curl_easy_perform(curl_handle);
/* check for errors */
if(res != CURLE_OK) {
fprintf(stderr, "curl_easy_perform() failed: %s\n",
/* cleanup curl stuff */
return chunk;
void download_image(IOTHUB_MODULE_CLIENT_LL_HANDLE iotHubModuleClientHandle)
ip_address_node * address_node = root_node;
struct MemoryStruct image = download_file(address_node->address);
if (image.size > 1)
IOTHUB_MESSAGE_HANDLE message_handle = IoTHubMessage_CreateFromByteArray((char*)image.memory, image.size);
MAP_HANDLE propMap = IoTHubMessage_Properties(message_handle);
Map_AddOrUpdate(propMap, "IpAddress", address_node->address);
IOTHUB_CLIENT_RESULT clientResult = IoTHubModuleClient_LL_SendEventToOutputAsync(iotHubModuleClientHandle, message_handle, "IncomingImage", NULL,NULL);
if (clientResult != IOTHUB_CLIENT_OK)
printf("IoTHubModuleClient_LL_SendEventToOutputAsync failed on sending to output IncomingImage, err=%d\n", clientResult);
address_node = address_node->next;
} while(address_node != NULL);

view raw


hosted with ❤ by GitHub

Downloading from the Audio feed

Now that the image feed is being published to the Edge Hub, its time to connect the audio feed. The audio feed is trickier since the Izon camera doesn’t have an easy to use endpoint (that I know of) for downloading the audio samples like we can with the image feed. In the next entry in this series, an Audio feed will be derived from an RSTP stream.


Using Protocol Buffers with Azure IoT Edge

Google’s Protocol Buffers are a perfect fit with the multilingual approach of Azure IoT Edge. Using ProtoBuf, a message format can be written once and used across multiple frameworks and languages while benefiting from the speed and message size intrinsic to ProtoBuf. For this Azure IoT Edge use case, we will generate a message in C++ and send it to a module written in Python to filter out the values that are sent to IoT Hub.


  • Create the message format
  • Create a C Azure IoT Edge Module
    • Add ProtoBuffers to build
    • Create C models
  • Create a Python Azure IoT Edge Module
    • Add ProtoBuffers to project
    • Create Python Module

Create the message format

Creating the message format is trivial. Following the language guide, there are two message types to create.

  1. A temperature reading, consisting of a float and a string
  2. An array of the previous reading with a string
syntax = "proto3";

message TemperatureReading {
  int reading = 1;
  string timestamp = 2;

message TemperatureReadingUpload {
  string uploaded_timestamp = 1;
  repeated TemperatureReading readings = 2;

The above is all that is needed to create the model for ProtoBuf. Creating the language specific code for each module is covered in their module sections.

Create a C Azure IoT Edge Module


This article assumes that you use a computer or virtual machine running Windows or Linux as your development machine. And you simulate your IoT Edge device on your development machine.


To create a module, you need Docker to build the module image, and a container registry to hold the module image:

Create a new solution template

Take these steps to create an IoT Edge module based on Azure IoT C SDK using Visual Studio Code and the Azure IoT Edge extension. First you create a solution, and then you generate the first module in that solution. Each solution can contain more than one module.

  1. In Visual Studio Code, select View > Integrated Terminal.
  2. Select View > Command Palette.
  3. In the command palette, enter and run the command Azure IoT Edge: New IoT Edge Solution.Run New IoT Edge Solution
  4. Browse to the folder where you want to create the new solution. Choose Select folder.
  5. Enter a name for your solution.
  6. Select C Module as the template for the first module in the solution.
  7. Enter a name for your module. Choose a name that’s unique within your container registry.
  8. Provide the name of the module’s image repository. VS Code autopopulates the module name with localhost:5000. Replace it with your own registry information. If you use a local Docker registry for testing, then localhost is fine. If you use Azure Container Registry, then use the login server from your registry’s settings. The login server looks like .azurecr.io.

VS Code takes the information you provided, creates an IoT Edge solution, and then loads it in a new window.

View IoT Edge solution

There are four items within the solution:

  • .vscode folder contains debug configurations.
  • modules folder has subfolders for each module. At this point, you only have one. But you can add more in the command palette with the command Azure IoT Edge: Add IoT Edge Module.
  • An .env file lists your environment variables. If Azure Container Registry is your registry, you’ll have an Azure Container Registry username and password in it.


    The environment file is only created if you provide an image repository for the module. If you accepted the localhost defaults to test and debug locally, then you don’t need to declare environment variables.

  • deployment.template.json file lists your new module along with a sample tempSensor module that simulates data you can use for testing. For more information about how deployment manifests work, see Learn how to use deployment manifests to deploy modules and establish routes.

Develop your module

The default C module code that comes with the solution is located at modules >> main.c. The module and the deployment.template.json file are set up so that you can build the solution, push it to your container registry, and deploy it to a device to start testing without touching any code. The module is built to simply take input from a source (in this case, the tempSensor module that simulates data) and pipe it to IoT Hub.

When you’re ready to customize the C template with your own code, use the Azure IoT Hub SDKs to build modules that address the key needs for IoT solutions such as security, device management, and reliability.

Compile the Protocol Buffer file

To compile the Protocol Buffer file, use the command line compiler protoc. For more information on how to use protoc for each platform, check out the Protocol Buffer documentation. For the C module, we will use the C++ compiler options:

protoc --proto_path=src --cpp_out=model src/temp.proto

To create and serialize the object, use the following code:

TemperatureReading reading;
reading.set_reading(get_temperature_reading()); //get_temperature_reading is your function on generating the temperature reading value
auto message_body = reading.SerializeAsString();


Build and deploy your module for debugging

In each module folder, there are several Docker files for different container types. Use any of these files that end with the extension .debug to build your module for testing. Currently, C modules support debugging only in Linux amd64 containers.

  1. In VS Code, navigate to the deployment.template.json file. Update your module image URL by adding .debug to the end.Add **.debug** to your image name
  2. Replace the C module createOptions in deployment.template.json with below content and save this file:
    "createOptions": "{\"HostConfig\": {\"Privileged\": true}}"
  3. In the VS Code command palette, enter and run the command Edge: Build IoT Edge solution.
  4. Select the deployment.template.json file for your solution from the command palette.
  5. In Azure IoT Hub Device Explorer, right-click an IoT Edge device ID. Then select Create deployment for Single device.
  6. Open your solution’s config folder. Then select the deployment.json file. Choose Select Edge Deployment Manifest.

You’ll see the deployment successfully created with a deployment ID in a VS Code-integrated terminal.

Check your container status in the VS Code Docker explorer or by running the docker ps command in the terminal.

Start debugging C module in VS Code

VS Code keeps debugging configuration information in a launch.json file located in a .vscode folder in your workspace. This launch.json file was generated when you created a new IoT Edge solution. It updates each time you add a new module that supports debugging.

  1. Navigate to the VS Code debug view. Select the debug configuration file for your module. The debug option name should be similar to ModuleName Remote Debug (C)Select debug configuration.
  2. Navigate to main.c. Add a breakpoint in this file.
  3. Select Start Debugging or select F5. Select the process to attach to.
  4. In VS Code Debug view, you’ll see the variables in the left panel.

The preceding example shows how to debug C IoT Edge modules on containers. It added exposed ports in your module container createOptions. After you finish debugging your C modules, we recommend you remove these exposed ports for production-ready IoT Edge modules.

Create a Python Azure IoT Edge Module

Create an IoT Edge module project

The following steps create an IoT Edge Python module by using Visual Studio Code and the Azure IoT Edge extension.

Create a new solution

Use the Python package cookiecutter to create a Python solution template that you can build on top of.

  1. In Visual Studio Code, select View > Integrated Terminal to open the VS Code integrated terminal.
  2. In the integrated terminal, enter the following command to install (or update) cookiecutter, which you use to create the IoT Edge solution template in VS Code:
    pip install --upgrade --user cookiecutter

    Ensure the directory where cookiecutter will be installed is in your environment’s Path in order to make it possible to invoke it from a command prompt.

  3. Select View > Command Palette to open the VS Code command palette.
  4. In the command palette, enter and run the command Azure: Sign in and follow the instructions to sign in your Azure account. If you’re already signed in, you can skip this step.
  5. In the command palette, enter and run the command Azure IoT Edge: New IoT Edge solution. In the command palette, provide the following information to create your solution:
    1. Select the folder where you want to create the solution.
    2. Provide a name for your solution or accept the default EdgeSolution.
    3. Choose Python Module as the module template.
    4. Name your module PythonModule.
    5. Specify the Azure container registry that you created in the previous section as the image repository for your first module. Replace localhost:5000 with the login server value that you copied. The final string looks like <registry name>.azurecr.io/pythonmodule.

The VS Code window loads your IoT Edge solution workspace: the modules folder, a deployment manifest template file, and a .env file.

Add your registry credentials

The environment file stores the credentials for your container repository and shares them with the IoT Edge runtime. The runtime needs these credentials to pull your private images onto the IoT Edge device.

  1. In the VS Code explorer, open the .env file.
  2. Update the fields with the username and password values that you copied from your Azure container registry.
  3. Save this file.

Compile the Protocol Buffer file

To compile the Protocol Buffer file, use the command line compiler protoc. For more information on how to use protoc for each platform, check out the Protocol Buffer documentation. For the Python module, we will use the Python compiler options:

protoc --proto_path=src --python_out=model src/temp.proto

Update the module with custom code

Each template includes sample code, which takes simulated sensor data from the tempSensor module and routes it to the IoT hub. In this section, add the code that expands the PythonModule to analyze the messages before sending them.

  1. In the VS Code explorer, open modules > PythonModule > main.py.
  2. At the top of the main.py file, import the temp_pb3 library that was created by protoc:
    import temp_pb3
  3. Add the TEMPERATURE_THRESHOLD and TWIN_CALLBACKS variables under the global counters. The temperature threshold sets the value that the measured machine temperature must exceed for the data to be sent to the IoT hub.
  4. Replace the receive_message_callback function with the following code:
    # receive_message_callback is invoked when an incoming message arrives on the specified 
    # input queue (in the case of this sample, "input1").  Because this is a filter module, 
    # we forward this message to the "output1" queue.
    def receive_message_callback(message, hubManager):
        global RECEIVE_CALLBACKS
        message_buffer = message.get_bytearray()
        map_properties = message.properties()
        key_value_pair = map_properties.get_internals()
        print ( "    Properties: %s" % key_value_pair )
        print ( "    Total calls received: %d" % RECEIVE_CALLBACKS )
        data = TemperatureReading.ParseFromString(message_buffer)
        if data.reading > TEMPERATURE_THRESHOLD:
            map_properties.add("MessageType", "Alert")
            print("Machine temperature %s exceeds threshold %s" % (data["machine"]["temperature"], TEMPERATURE_THRESHOLD))
        hubManager.forward_event_to_output("output1", message, 0)
        return IoTHubMessageDispositionResult.ACCEPTED
  5. Add a new function called module_twin_callback. This function is invoked when the desired properties are updated.
    # module_twin_callback is invoked when the module twin's desired properties are updated.
    def module_twin_callback(update_state, payload, user_context):
        global TWIN_CALLBACKS
        print ( "\nTwin callback called with:\nupdateStatus = %s\npayload = %s\ncontext = %s" % (update_state, payload, user_context) )
        data = json.loads(payload)
        if "desired" in data and "TemperatureThreshold" in data["desired"]:
            TEMPERATURE_THRESHOLD = data["desired"]["TemperatureThreshold"]
        if "TemperatureThreshold" in data:
            TEMPERATURE_THRESHOLD = data["TemperatureThreshold"]
        TWIN_CALLBACKS += 1
        print ( "Total calls confirmed: %d\n" % TWIN_CALLBACKS )
  6. In the HubManager class, add a new line to the init method to initialize the module_twin_callback function that you just added:
    # Sets the callback when a module twin's desired properties are updated.
    self.client.set_module_twin_callback(module_twin_callback, self)
  7. Save this file.

Build your IoT Edge solution

In the previous section, you created an IoT Edge solution and added code to the PythonModule to filter out messages where the reported machine temperature is below the acceptable threshold. Now you need to build the solution as a container image and push it to your container registry.

  1. Sign in to Docker by entering the following command in the Visual Studio Code integrated terminal. Then you can push your module image to your Azure container registry:
    docker login -u <ACR username> -p <ACR password> <ACR login server>

    Use the username, password, and login server that you copied from your Azure container registry in the first section. You can also retrieve these values from the Access keys section of your registry in the Azure portal.

  2. In the VS Code explorer, open the deployment.template.json file in your IoT Edge solution workspace.This file tells the $edgeAgent to deploy two modules: tempSensor, which simulates device data, and PythonModule. The PythonModule.image value is set to a Linux amd64 version of the image. To learn more about deployment manifests, see Understand how IoT Edge modules can be used, configured, and reused.This file also contains your registry credentials. In the template file, your user name and password are filled in with placeholders. When you generate the deployment manifest, the fields are updated with the values that you added to the .env file.
  3. Add the PythonModule module twin to the deployment manifest. Insert the following JSON content at the bottom of the moduleContent section, after the $edgeHub module twin:
        "PythonModule": {
  4. Save this file.
  5. In the VS Code explorer, right-click the deployment.template.json file and select Build and Push IoT Edge solution.

When you tell Visual Studio Code to build your solution, it first takes the information in the deployment template and generates a deployment.json file in a new folder named config. Then it runs two commands in the integrated terminal: docker build and docker push. These two commands build your code, containerize the Python code, and then push the code to the container registry that you specified when you initialized the solution.

You can see the full container image address with tag in the docker build command that runs in the VS Code integrated terminal. The image address is built from information in the module.json file with the format <repository>:<version>-<platform>. For this tutorial, it should look like registryname.azurecr.io/pythonmodule:0.0.1-amd64.

Deploy and run the solution

You can use the Azure portal to deploy your Python module to an IoT Edge device like you did in the quickstarts. You can also deploy and monitor modules from within Visual Studio Code. The following sections use the Azure IoT Edge extension for VS Code that was listed in the prerequisites. Install the extension now, if you didn’t already.

  1. Open the VS Code command palette by selecting View > Command Palette.
  2. Search for and run the command Azure: Sign in. Follow the instructions to sign in your Azure account.
  3. In the command palette, search for and run the command Azure IoT Hub: Select IoT Hub.
  4. Select the subscription that contains your IoT hub, and then select the IoT hub that you want to access.
  5. In the VS Code explorer, expand the Azure IoT Hub Devices section.
  6. Right-click the name of your IoT Edge device, and then select Create Deployment for IoT Edge device.
  7. Browse to the solution folder that contains the PythonModule. Open the config folder, select the deployment.json file, and then choose Select Edge Deployment Manifest.
  8. Refresh the Azure IoT Hub Devices section. You should see the new PythonModule running along with the TempSensor module and the $edgeAgent and $edgeHub.

Azure IoT Edge – exec user process caused “exec format error”

If running Edge on a Raspberry Pi  and an Edge container’s logs show  ‘exec user process caused “exec format error”‘ as an error then most likely you are running a non Raspberry Pi container on the Raspberry Pi. If the docker file used to build the container starts with:

  • FROM microsoft/dotnet:2.0.0-runtime


  • FROM microsoft/dotnet:2.0.0-runtime-nanoserver-1709

then the line above should be changed to one of the following:

  • FROM microsoft/dotnet:2.0.5-runtime-stretch-arm32v7
  • FROM microsoft/dotnet:2.0-runtime-stretch-arm32v7
  • FROM microsoft/dotnet:2.0.5-runtime-deps-stretch-arm32v7
  • FROM microsoft/dotnet:2.0-runtime-deps-stretch-arm32v7

Azure IoT Edge stuck restarting

NOTE: This is for the private preview version of Azure IoT Edge

If your Azure IoT Edge runtime is giving the status of:

ERROR: Runtime is restarting. Please retry later.

and is not allowing for any of the other commands and you have waited an appropriated amount of time, then try the following:

First stop the edgeAgent container in docker:

sudo docker stop edgeAgent

Then, run the setup script for the iotedgectl again:

sudo iotedgectl setup --connection-string "{device connection string}" --auto-cert-gen-force-no-passwords (make sure to use your parameters)

The edge run-time should restart appropriately.