Authoring for Pluralsight – Developing Microsoft Azure Intelligent Edge Solutions

Off to start another course for Pluralsight. This time its Developing Microsoft Azure Intelligent Edge Solutions. If you would like to check out any of my other courses, visit my author’s profile. The new course will cover the following topics:

  • Edge
    • IoT Architecture
    • IoT use cases and solutions
    • Edge Architecture
  • Azure IoT Hub
    • Overview of the IoT Ecosystem in Azure
    • IoT Hub message routing
    • Stream processing overview
  • Hot, Warm, and Cold paths
    • Use cases for hot, warm, and cold paths
    • Hot path with event hubs and log app
    • Warm path with Cosmos DB
    • Cold path with Azure Blob Storage
  • Real Time and Batch Processing
    • Overview and Demos of Stream Analytics Service
    • Overview and Demos of Time Series Insights

Pluralsight Course Published – Designing an Intelligent Edge in Microsoft Azure

Designing an Intelligent Edge in Microsoft Azure was just published on Pluralsight! Check it out. Here is a synopsis of what’s in it:

This course targets software developers that are looking to integrate AI solutions in edge scenarios ranging from an edge data center down to secure microcontrollers. This course will showcase how to design solutions using Microsoft Azure.

Cloud computing has moved more and more out of the cloud and onto the edge. In this course, Designing an Intelligent Edge in Microsoft Azure, you will learn foundational knowledge of edge computing, its intersection with AI, and how to utilize both with Microsoft Azure. First, you will learn the concepts of edge computing. Next, you will discover how to create an edge solution utilizing Azure Stack, Azure Data Box Edge, and Azure IoT Edge. Finally, you will explore how to utilize off the shelf AI and build your own for Azure IoT Edge. When you are finished with this course, you will have the skills and knowledge of AI on the edge needed to architect your next edge solution. Software required: Microsoft Azure, .NET

 

iotedge: error while loading shared libraries: libssl.so.1.0.2: cannot open shared object file: No such file or directory – Raspberry Pi

After installing Azure IoT Edge using the guide for Linux ARM32, the following error was presented: “iotedge: error while loading shared libraries: libssl.so.1.0.2: cannot open shared object file: No such file or directory“. 

The fix was simple enough, just install the building libssl1.02 using the following command:

sudo apt-get install libssl1.0.2

Test by running the iotedge command:

iotedge

azureiotedgeCapture.PNG

If that works successfully, restart the iotedge service:

service iotedge edge restart

Verify that it is running by checking the service status:

service iotedge edge status

azureiotedgeCapture

Azure IoT Edge – YOLO, Stream Analytics Service, and Blob Storage

As a continuation of the Izon camera hack //TODO: link to previous article, I wanted to detect if my dog was using the doggy door in the main room. The approach was going to be simple at first, detect the dog in the room and not in the room. When in the room changes (or not in the room), upload 10 seconds worth of images to Azure to see if the dog used the door.

Image Classification

To detect the dog, the first and largest challenge to these types of tasks is getting enough images to train the model. For me, this meant saving images of the dog in a pre-aligned shot. This is easy enough to accomplish; the room the images will be processed in should only have the dog moving in it. Since he is the only moving object, YOLO can be used to detect the position of objects in the room and then the position of these objects can be checked to see if there is any movement. If there is any movement, the images can be saved for later categorization. To accomplish this, there will be four modules:

  • Camera Module – Accesses the camera feeds to save the images
  • Object Detection Module – Uses YOLO to detect object and object positions
  • Motion Detection Module – Uses Stream Analytics Service to detect if object positions are moving.
  • Image Storage Module – Uses Blob Storage so save and delete the images

Module Arch

The Camera module will send the timestamped images to the Object Detection Module and the Image Storage Module. The Object Detection Module will then use YOLO to detect the objects and their positions in the image. Those detection results will be sent to the Motion Detection Module, which will use Streaming Analytics Service to see if there was motion detected over the last ten seconds. If there is no motion detected over the last ten seconds, then the Motion Detection Module will send a delete command to the Image Storage Module to remove the image without motion from the store. The routing Table will look as so:

These modules will be broken up into their own articles for readability and searchability. If there is no link to a module article it is because that article is not completed or is not published yet.

Upcoming Pluralsight Course – Designing an Intelligent Edge in Microsoft Azure

Off to start another course for Pluralsight. This time its Designing an Intelligent Edge in Microsoft Azure. If you would like to check out any of my other courses, visit my author’s profile. The new course will cover the following topics:

  • Edge –
    • Scenarios
    • Concerns
    • Architecture
  • Azure AI Pipelines – Overview with edge
  • Edge Pipelines –
    • Azure Stack
    • Azure Databox Edge
    • Azure IoT Edge
  • Cognitive Services – Overview with Edge
  • Azure Databricks – Overview
  • Azure Machine Learning VMs
  • Project Brainwave

Speaking at DotNetSouth.Tech

I look forward to speaking on AI on the Edge at DotNetSouth.Tech. This year is the conference’s first year so check it out.

AI on the Edge

The next evolution in cloud computing is a smarter application not in the cloud. As the cloud has continued to evolve, the applications that utilize it have had more and more capabilities of the cloud. This presentation will show how to push logic and machine learning from the cloud to an edge application. Afterward, creating edge applications which utilize the intelligence of the cloud should become effortless.

 

Speaking at Orlando Code Camp

I am happy to announce I will speaking at the Orlando Code Camp again this year. I will be presenting AI on the Edge, a look into Microsoft’s Azure IoT Edge.

Title

AI on the Edge

Description

The next evolution in cloud computing is a smarter application not in the cloud. As the cloud has continued to evolve, the applications that utilize it have had more and more capabilities of the cloud. This presentation will show how to push logic and machine learning from the cloud to an edge application. Afterward, creating edge applications which utilize the intelligence of the cloud should become effortless.

Hacking Izon Cameras and using Azure IoT Edge

After Izon announced that they were closing down their services (leaving the cameras I already owned useless), I decided to turn them into something useful using Azure. First let me list some resources:

Use the Will it hack link to get access to the mobileye website and verify that the Izon device is still streaming and still working. If it is working, you are already done with edits to the device unless you would like to change the passwords (which you should).

Our goals are as follows:

  • Process the video feed from the Izon camera (we will cheat this early on and only use the image feed)
    • Check for motion
    • Check for faces
    • Check if faces are white listed
    • Check for my dog
  • Process the audio feed
    • Check for any noise
    • Check for non human noises
    • Check for dog barks
    • Check for my and my wife’s voice

These are all stretch goals that will be referred back to as the project moves forward.

Create the Azure IoT Edge module

For the first module, we will use the C Module base image. We are looking for two things from this module:

  • Download the picture feed and pass it to the Edge Hub
  • Download the audio feed and pass it to the Edge Hub

If you don’t know where to get started with the C module of the Azure IoT Edge platform, there is helpful information on the Azure Documentation page. Once the C module is created and ready for editing, we are going to connect to the image feed from the devices. To make this simple, both feeds will be retrieved using HTTP. For the video feed, its simple enough to grab images from the Izon camera existing camera feed.

Now one thing we need, is to be able to connect to each camera within the local network shared with the Edge. Since we would like to be able to add and remove cameras, we will use the device twin to update and manage the list of IP address. The code for updating the list is as follows:

With that code in place, the list of IP addresses can be updated from the Azure UI and the Azure Service SDKs.

Downloading from the Image feed

The Izon cameras make downloading the image feed trivial. There is an existing endpoint where you can grab the latest image directly from the camera’s web server. The latest image is always at /cgi-bin/img-d1.cgi. (NOTE: if you are checking this image from a browser, be sure to have some cache busting!). To download this image into our module, we will use the Curl library for it’s easy HTTP implementation. To add Curl to our Edge module, we will add the following lines to the Dockerfile.amd64.debug:

With curl now added to the image, it can be utilized in code by adding it to the method invoked in our main loop. The code will download the file for each entry in the IP address list. Once the image is downloaded, it will send it as a message to the Edge Hub and add the IP address of the camera to the message header. Here is that code:

Downloading from the Audio feed

Now that the image feed is being published to the Edge Hub, its time to connect the audio feed. The audio feed is trickier since the Izon camera doesn’t have an easy to use endpoint (that I know of) for downloading the audio samples like we can with the image feed. In the next entry in this series, an Audio feed will be derived from an RSTP stream.

 

Using Open CV C++ with Azure IoT Edge

If you are looking for a guide on creating an Open CV module in Python, check out a guide here. This guide will focus on creating an Azure IoT Edge module in C++. To accomplish this we need to take the following steps:

Create the Azure IoT Edge Module

Prerequisites

This article assumes that you use a computer or virtual machine running Windows or Linux as your development machine. And you simulate your IoT Edge device on your development machine.

Needs:

To create a module, you need Docker to build the module image, and a container registry to hold the module image:

Create a new solution template

Take these steps to create an IoT Edge module based on Azure IoT C SDK using Visual Studio Code and the Azure IoT Edge extension. First you create a solution, and then you generate the first module in that solution. Each solution can contain more than one module.

  1. In Visual Studio Code, select View > Integrated Terminal.
  2. Select View > Command Palette.
  3. In the command palette, enter and run the command Azure IoT Edge: New IoT Edge Solution.Run New IoT Edge Solution
  4. Browse to the folder where you want to create the new solution. Choose Select folder.
  5. Enter a name for your solution.
  6. Select C Module as the template for the first module in the solution.
  7. Enter a name for your module. Choose a name that’s unique within your container registry.
  8. Provide the name of the module’s image repository. VS Code autopopulates the module name with localhost:5000. Replace it with your own registry information. If you use a local Docker registry for testing, then localhost is fine. If you use Azure Container Registry, then use the login server from your registry’s settings. The login server looks like .azurecr.io.

VS Code takes the information you provided, creates an IoT Edge solution, and then loads it in a new window.

View IoT Edge solution

There are four items within the solution:

  • .vscode folder contains debug configurations.
  • modules folder has subfolders for each module. At this point, you only have one. But you can add more in the command palette with the command Azure IoT Edge: Add IoT Edge Module.
  • An .env file lists your environment variables. If Azure Container Registry is your registry, you’ll have an Azure Container Registry username and password in it.

    Note

    The environment file is only created if you provide an image repository for the module. If you accepted the localhost defaults to test and debug locally, then you don’t need to declare environment variables.

  • deployment.template.json file lists your new module along with a sample tempSensor module that simulates data you can use for testing. For more information about how deployment manifests work, see Learn how to use deployment manifests to deploy modules and establish routes.

Develop your module

The default C module code that comes with the solution is located at modules >  > main.c. The module and the deployment.template.json file are set up so that you can build the solution, push it to your container registry, and deploy it to a device to start testing without touching any code. The module is built to simply take input from a source (in this case, the tempSensor module that simulates data) and pipe it to IoT Hub.

When you’re ready to customize the C template with your own code, use the Azure IoT Hub SDKs to build modules that address the key needs for IoT solutions such as security, device management, and reliability.

Build and deploy your module for debugging

In each module folder, there are several Docker files for different container types. Use any of these files that end with the extension .debug to build your module for testing. Currently, C modules support debugging only in Linux amd64 containers.

  1. In VS Code, navigate to the deployment.template.json file. Update your module image URL by adding .debug to the end.Add **.debug** to your image name
  2. Replace the Node.js module createOptions in deployment.template.json with below content and save this file:
    "createOptions": "{\"HostConfig\": {\"Privileged\": true}}"
    
  3. In the VS Code command palette, enter and run the command Edge: Build IoT Edge solution.
  4. Select the deployment.template.json file for your solution from the command palette.
  5. In Azure IoT Hub Device Explorer, right-click an IoT Edge device ID. Then select Create deployment for IoT Edge device.
  6. Open your solution’s config folder. Then select the deployment.json file. Choose Select Edge Deployment Manifest.

You’ll see the deployment successfully created with a deployment ID in a VS Code-integrated terminal.

Check your container status in the VS Code Docker explorer or by running the docker ps command in the terminal.

Start debugging C module in VS Code

VS Code keeps debugging configuration information in a launch.json file located in a .vscode folder in your workspace. This launch.json file was generated when you created a new IoT Edge solution. It updates each time you add a new module that supports debugging.

  1. Navigate to the VS Code debug view. Select the debug configuration file for your module. The debug option name should be similar to ModuleName Remote Debug (C)Select debug configuration.
  2. Navigate to main.c. Add a breakpoint in this file.
  3. Select Start Debugging or select F5. Select the process to attach to.
  4. In VS Code Debug view, you’ll see the variables in the left panel.

The preceding example shows how to debug C IoT Edge modules on containers. It added exposed ports in your module container createOptions. After you finish debugging your Node.js modules, we recommend you remove these exposed ports for production-ready IoT Edge modules.

Create a working Open CV Build

The working environment is an Ubuntu 18.04 64 bit Desktop OS running Clion using an embedded version of CMake 3.10. Open CV is added via source as a submodule to the project and added as a package in the CMakeLists.txt with the following line:

FIND_PACKAGE (OpenCV REQUIRED)

Once that was added to the CMakeLists.txt, the main.cpp file was changed to the following code:

Deploy the Azure IoT Edge Module

Once you create IoT Edge modules with your business logic, you want to deploy them to your devices to operate at the edge. If you have multiple modules that work together to collect and process data, you can deploy them all at once and declare the routing rules that connect them.

This article shows how to create a JSON deployment manifest, then use that file to push the deployment to an IoT Edge device. For information about creating a deployment that targets multiple devices based on their shared tags, see Deploy and monitor IoT Edge modules at scale

Prerequisites

Configure a deployment manifest

A deployment manifest is a JSON document that describes which modules to deploy, how data flows between the modules, and desired properties of the module twins. For more information about how deployment manifests work and how to create them, see Understand how IoT Edge modules can be used, configured, and reused.

To deploy modules using Visual Studio Code, save the deployment manifest locally as a .JSON file. You will use the file path in the next section when you run the command to apply the configuration to your device.

Here’s a basic deployment manifest with one module as an example:

{
  "modulesContent": {
    "$edgeAgent": {
      "properties.desired": {
        "schemaVersion": "1.0",
        "runtime": {
          "type": "docker",
          "settings": {
            "minDockerVersion": "v1.25",
            "loggingOptions": "",
            "registryCredentials": {}
          }
        },
        "systemModules": {
          "edgeAgent": {
            "type": "docker",
            "settings": {
              "image": "mcr.microsoft.com/azureiotedge-agent:1.0",
              "createOptions": "{}"
            }
          },
          "edgeHub": {
            "type": "docker",
            "status": "running",
            "restartPolicy": "always",
            "settings": {
              "image": "mcr.microsoft.com/azureiotedge-hub:1.0",
              "createOptions": "{}"
            }
          }
        },
        "modules": {
          "tempSensor": {
            "version": "1.0",
            "type": "docker",
            "status": "running",
            "restartPolicy": "always",
            "settings": {
              "image": "mcr.microsoft.com/azureiotedge-simulated-temperature-sensor:1.0",
              "createOptions": "{}"
            }
          }
        }
      }
    },
    "$edgeHub": {
      "properties.desired": {
        "schemaVersion": "1.0",
        "routes": {
            "route": "FROM /* INTO $upstream"
        },
        "storeAndForwardConfiguration": {
          "timeToLiveSecs": 7200
        }
      }
    },
    "tempSensor": {
      "properties.desired": {}
    }
  }
}

Sign in to access your IoT hub

You can use the Azure IoT extensions for Visual Studio Code to perform operations with your IoT hub. For these operations to work, you need to sign in to your Azure account and select the IoT hub that you are working on.

  1. In Visual Studio Code, open the Explorer view.
  2. At the bottom of the Explorer, expand the Azure IoT Hub Devices section.Expand Azure IoT Hub Devices
  3. Click on the  in the Azure IoT Hub Devices section header. If you don’t see the ellipsis, hover over the header.
  4. Choose Select IoT Hub.
  5. If you are not signed in to your Azure account, follow the prompts to do so.
  6. Select your Azure subscription.
  7. Select your IoT hub.

Deploy to your device

You deploy modules to your device by applying the deployment manifest that you configured with the module information.

  1. In the Visual Studio Code explorer view, expand the Azure IoT Hub Devices section.
  2. Right-click on the device that you want to configure with the deployment manifest.
  3. Select Create Deployment for IoT Edge Device.
  4. Navigate to the deployment manifest JSON file that you want to use, and click Select Edge Deployment Manifest.Select Edge Deployment Manifest

The results of your deployment are printed in the VS Code output. Successful deployments are applied within a few minutes if the target device is running and connected to the internet.

View modules on your device

Once you’ve deployed modules to your device, you can view all of them in the Azure IoT Hub Devices section. Select the arrow next to your IoT Edge device to expand it. All the currently running modules are displayed.

If you recently deployed new modules to a device, hover over the Azure IoT Hub Devices section header and select the refresh icon to update the view.

Right-click the name of a module to view and edit the module twin.