Using Open CV C++ with Azure IoT Edge

If you are looking for a guide on creating an Open CV module in Python, check out a guide here. This guide will focus on creating an Azure IoT Edge module in C++. To accomplish this we need to take the following steps:

Create the Azure IoT Edge Module

Prerequisites

This article assumes that you use a computer or virtual machine running Windows or Linux as your development machine. And you simulate your IoT Edge device on your development machine.

Needs:

To create a module, you need Docker to build the module image, and a container registry to hold the module image:

Create a new solution template

Take these steps to create an IoT Edge module based on Azure IoT C SDK using Visual Studio Code and the Azure IoT Edge extension. First you create a solution, and then you generate the first module in that solution. Each solution can contain more than one module.

  1. In Visual Studio Code, select View > Integrated Terminal.
  2. Select View > Command Palette.
  3. In the command palette, enter and run the command Azure IoT Edge: New IoT Edge Solution.Run New IoT Edge Solution
  4. Browse to the folder where you want to create the new solution. Choose Select folder.
  5. Enter a name for your solution.
  6. Select C Module as the template for the first module in the solution.
  7. Enter a name for your module. Choose a name that’s unique within your container registry.
  8. Provide the name of the module’s image repository. VS Code autopopulates the module name with localhost:5000. Replace it with your own registry information. If you use a local Docker registry for testing, then localhost is fine. If you use Azure Container Registry, then use the login server from your registry’s settings. The login server looks like .azurecr.io.

VS Code takes the information you provided, creates an IoT Edge solution, and then loads it in a new window.

View IoT Edge solution

There are four items within the solution:

  • .vscode folder contains debug configurations.
  • modules folder has subfolders for each module. At this point, you only have one. But you can add more in the command palette with the command Azure IoT Edge: Add IoT Edge Module.
  • An .env file lists your environment variables. If Azure Container Registry is your registry, you’ll have an Azure Container Registry username and password in it.

    Note

    The environment file is only created if you provide an image repository for the module. If you accepted the localhost defaults to test and debug locally, then you don’t need to declare environment variables.

  • deployment.template.json file lists your new module along with a sample tempSensor module that simulates data you can use for testing. For more information about how deployment manifests work, see Learn how to use deployment manifests to deploy modules and establish routes.

Develop your module

The default C module code that comes with the solution is located at modules >  > main.c. The module and the deployment.template.json file are set up so that you can build the solution, push it to your container registry, and deploy it to a device to start testing without touching any code. The module is built to simply take input from a source (in this case, the tempSensor module that simulates data) and pipe it to IoT Hub.

When you’re ready to customize the C template with your own code, use the Azure IoT Hub SDKs to build modules that address the key needs for IoT solutions such as security, device management, and reliability.

Build and deploy your module for debugging

In each module folder, there are several Docker files for different container types. Use any of these files that end with the extension .debug to build your module for testing. Currently, C modules support debugging only in Linux amd64 containers.

  1. In VS Code, navigate to the deployment.template.json file. Update your module image URL by adding .debug to the end.Add **.debug** to your image name
  2. Replace the Node.js module createOptions in deployment.template.json with below content and save this file:
    "createOptions": "{\"HostConfig\": {\"Privileged\": true}}"
    
  3. In the VS Code command palette, enter and run the command Edge: Build IoT Edge solution.
  4. Select the deployment.template.json file for your solution from the command palette.
  5. In Azure IoT Hub Device Explorer, right-click an IoT Edge device ID. Then select Create deployment for IoT Edge device.
  6. Open your solution’s config folder. Then select the deployment.json file. Choose Select Edge Deployment Manifest.

You’ll see the deployment successfully created with a deployment ID in a VS Code-integrated terminal.

Check your container status in the VS Code Docker explorer or by running the docker ps command in the terminal.

Start debugging C module in VS Code

VS Code keeps debugging configuration information in a launch.json file located in a .vscode folder in your workspace. This launch.json file was generated when you created a new IoT Edge solution. It updates each time you add a new module that supports debugging.

  1. Navigate to the VS Code debug view. Select the debug configuration file for your module. The debug option name should be similar to ModuleName Remote Debug (C)Select debug configuration.
  2. Navigate to main.c. Add a breakpoint in this file.
  3. Select Start Debugging or select F5. Select the process to attach to.
  4. In VS Code Debug view, you’ll see the variables in the left panel.

The preceding example shows how to debug C IoT Edge modules on containers. It added exposed ports in your module container createOptions. After you finish debugging your Node.js modules, we recommend you remove these exposed ports for production-ready IoT Edge modules.

Create a working Open CV Build

The working environment is an Ubuntu 18.04 64 bit Desktop OS running Clion using an embedded version of CMake 3.10. Open CV is added via source as a submodule to the project and added as a package in the CMakeLists.txt with the following line:

FIND_PACKAGE (OpenCV REQUIRED)

Once that was added to the CMakeLists.txt, the main.cpp file was changed to the following code:

Deploy the Azure IoT Edge Module

Once you create IoT Edge modules with your business logic, you want to deploy them to your devices to operate at the edge. If you have multiple modules that work together to collect and process data, you can deploy them all at once and declare the routing rules that connect them.

This article shows how to create a JSON deployment manifest, then use that file to push the deployment to an IoT Edge device. For information about creating a deployment that targets multiple devices based on their shared tags, see Deploy and monitor IoT Edge modules at scale

Prerequisites

Configure a deployment manifest

A deployment manifest is a JSON document that describes which modules to deploy, how data flows between the modules, and desired properties of the module twins. For more information about how deployment manifests work and how to create them, see Understand how IoT Edge modules can be used, configured, and reused.

To deploy modules using Visual Studio Code, save the deployment manifest locally as a .JSON file. You will use the file path in the next section when you run the command to apply the configuration to your device.

Here’s a basic deployment manifest with one module as an example:

{
  "modulesContent": {
    "$edgeAgent": {
      "properties.desired": {
        "schemaVersion": "1.0",
        "runtime": {
          "type": "docker",
          "settings": {
            "minDockerVersion": "v1.25",
            "loggingOptions": "",
            "registryCredentials": {}
          }
        },
        "systemModules": {
          "edgeAgent": {
            "type": "docker",
            "settings": {
              "image": "mcr.microsoft.com/azureiotedge-agent:1.0",
              "createOptions": "{}"
            }
          },
          "edgeHub": {
            "type": "docker",
            "status": "running",
            "restartPolicy": "always",
            "settings": {
              "image": "mcr.microsoft.com/azureiotedge-hub:1.0",
              "createOptions": "{}"
            }
          }
        },
        "modules": {
          "tempSensor": {
            "version": "1.0",
            "type": "docker",
            "status": "running",
            "restartPolicy": "always",
            "settings": {
              "image": "mcr.microsoft.com/azureiotedge-simulated-temperature-sensor:1.0",
              "createOptions": "{}"
            }
          }
        }
      }
    },
    "$edgeHub": {
      "properties.desired": {
        "schemaVersion": "1.0",
        "routes": {
            "route": "FROM /* INTO $upstream"
        },
        "storeAndForwardConfiguration": {
          "timeToLiveSecs": 7200
        }
      }
    },
    "tempSensor": {
      "properties.desired": {}
    }
  }
}

Sign in to access your IoT hub

You can use the Azure IoT extensions for Visual Studio Code to perform operations with your IoT hub. For these operations to work, you need to sign in to your Azure account and select the IoT hub that you are working on.

  1. In Visual Studio Code, open the Explorer view.
  2. At the bottom of the Explorer, expand the Azure IoT Hub Devices section.Expand Azure IoT Hub Devices
  3. Click on the  in the Azure IoT Hub Devices section header. If you don’t see the ellipsis, hover over the header.
  4. Choose Select IoT Hub.
  5. If you are not signed in to your Azure account, follow the prompts to do so.
  6. Select your Azure subscription.
  7. Select your IoT hub.

Deploy to your device

You deploy modules to your device by applying the deployment manifest that you configured with the module information.

  1. In the Visual Studio Code explorer view, expand the Azure IoT Hub Devices section.
  2. Right-click on the device that you want to configure with the deployment manifest.
  3. Select Create Deployment for IoT Edge Device.
  4. Navigate to the deployment manifest JSON file that you want to use, and click Select Edge Deployment Manifest.Select Edge Deployment Manifest

The results of your deployment are printed in the VS Code output. Successful deployments are applied within a few minutes if the target device is running and connected to the internet.

View modules on your device

Once you’ve deployed modules to your device, you can view all of them in the Azure IoT Hub Devices section. Select the arrow next to your IoT Edge device to expand it. All the currently running modules are displayed.

If you recently deployed new modules to a device, hover over the Azure IoT Hub Devices section header and select the refresh icon to update the view.

Right-click the name of a module to view and edit the module twin.

Pluralsight Course is out!

My Pluralsight course, Identifying Existing Products, Services, and Technologies in Use for Microsoft Azure, is out an available here. Check it out, here is the short and long descriptions:

Short description:
Microsoft Azure can host almost any application, but understanding how to use it with existing workflows is a must. In this course, you will learn how to integrate existing workflows, technologies, and processes with Microsoft Azure.
 
Long description:
Knowing how to integrate Microsoft Azure with an existing app’s workflow is essential to using Azure to host that application. In this course, Identifying Existing Products, Services, and Technologies in Use for Microsoft Azure, you will learn foundational knowledge of and gain the ability to navigate the Microsoft Azure documentation and utilize the tools for Microsoft Azure. First, you will discover how to navigate through the Microsoft Azure documentation. Next, you will learn how to utilize the different guides and tutorials of the Microsoft Azure products. Finally, you will explore how to work with Microsoft Azure using your existing tools and workflows. When you are finished with this course, you will have the skills and knowledge of Microsoft Azure tools and documentation needed to use the products, services, and technologies provided. 

South Florida Code Camp – Azure IoT Overview

March 2nd 2019, I will be presenting Azure IoT Overview at the South Florida Code Camp in Davie, FL. You can register here – and its FREE. Here is the synopsis of the presentation:

Abstract

Keeping up to date on all the new services and features for an entire cloud portfolio could be a full-time job. In this presentation, we will look at the state of IoT in Microsoft Azure and discuss how the different services work together to implement an enterprise solution. Use this presentation to get an overview of architecture and products so that the next time you are presented with an IoT problem in Azure you know the solution.

 

 

Authoring for Pluralsight

Coming soon I will be authoring a course for Pluralsight titled – “Identify Existing Products, Services and Technologies in Use For Microsoft Azure” . This course targets software developers who are looking to get started with Microsoft Azure services to build modern cloud-enabled solutions and want to further extend their knowledge of those services by learning how to use existing products, services, and technologies offered by Microsoft Azure.

Microsoft Azure is a host for almost any application, but determining how to use it within existing workflows is paramount for success. In this course, Identify Existing Products, Services and Technologies in Use, you will learn how to integrate existing workflows, technologies, and processes with Microsoft Azure.

We explore Microsoft Azure with the following technologies:

  • Languages, Frameworks, and IDEs –
    • IntelliJ IDEA
    • WebStorm
    • Visual Studio Code
    • .NET Core
    • C#
    • Java
    • JavaScript
    • Spring
    • NodeJS
    • Docker
  • Microsoft Azure Products
    • Azure App Services
    • Azure Kubernetes
    • Azure Functions
    • Azure IoT Hub

Hopefully we can take a developer familiar with the languages, frameworks, and ides available and make have them up and running on Microsoft Azure after this short course.

Quicken Loans TechCon 2018

September 20th at Cobb Center in Detroit I will be presenting:

Alternative Device Interfaces and Machine Learning

Abstract

In this presentation, we will look at the how users interface with machines without the use of touch. These different types of interaction have their benefits and pitfalls. To showcase the power of these user interactions we will explore: Voice commands with mobile applications, Speech Recognition, and Computer Vision. After this presentation, attendees will have the knowledge to create applications that can utilize voice, video, and machine learning.

Description

Users use voice (Alexa, Cortana, Google Now) or video as a mode of interaction with applications. More than a fad, this is a natural interface for users and is becoming more and more common with the ever-decreasing size of hardware.

Different types of interaction have their benefits and pitfalls. To showcase the power of these user interactions we will explore: Voice commands with two app types: UWP and Xamarin Forms (iOS and Android). Speech Recognition with Cognitive Services: Verifying the speaker with Speaker Recognition API. Computer Vision with Cognitive Services: Verifying a user with Face API.

By utilizing UWP, Xamarin, and Cognitive services; a device with the ultimate in customization for user interactions will be created. Come and see how!

Update – Techbash

UPDATE:

Another one of my talks was selected for Techbash: Alternative Device Interfaces and Machine Learning.

In this presentation, we will look at the how users interface with machines without the use of touch. These different types of interaction have their benefits and pitfalls. To showcase the power of these user interactions we will explore: Voice commands with mobile applications, Speech Recognition, and Computer Vision. After this presentation, attendees will have the knowledge to create applications that can utilize voice, video, and machine learning.

Users use voice (Alexa, Cortana, Google Now) or video as a mode of interaction with applications. More than a fad, this is a natural interface for users and is becoming more and more common with the ever-decreasing size of hardware.

Different types of interaction have their benefits and pitfalls. To showcase the power of these user interactions we will explore: Voice commands with two app types: UWP and Xamarin Forms (iOS and Android). Speech Recognition with Cognitive Services: Verifying the speaker with Speaker Recognition API. Computer Vision with Cognitive Services: Verifying a user with Face API.

By utilizing UWP, Xamarin, and Cognitive services; a device with the ultimate in customization for user interactions will be created. Come and see how!

Original:

This year I will be presenting Enable IoT with Edge Computing and Machine Learning at TechBash. Here is the outline:

Being able to run compute cycles on local hardware is a practice predating silicon circuits. Mobile and Web technology has pushed computation away from local hardware and onto remote servers. As prices in the cloud have decreased, more and more of the remote servers have moved there. This technology cycle is coming full circle with pushing the computation that would be done in the cloud down to the client. The catalyst for the cycle completing is latency and cost. Running computations on local hardware softens the load in the cloud and reduces overall cost and architectural complexity.

The difference now is how the computational logic is sent to the device. As of now, we rely on app stores and browsers to deliver the logic the client will use. Delivery mechanisms are evolving into writing code once and having the ability to run that logic in the cloud and push that logic to the client through your application and have that logic run on the device. In this presentation, we will look at how to accomplish this with existing Azure technologies and how to prepare for upcoming technologies to run these workloads.

 

Hosting a permanent redirect in Azure

WordPress charges for each domain attached to a blog. That is a bit much when you consider .net, .org, .whatever domains that are used on top of the .com domain most use. To get the use out of a single domain, a permanent redirect can be used.  Since Azure has a fixed price, invariant of how many domains you host there; it can be used for the permanent redirect.

Prerequisites

To complete this tutorial:

  • Create an App Service app, or use an app that you created for another tutorial.
  • Purchase a domain name and make sure you have access to the DNS registry for your domain provider (such as GoDaddy).

    For example, to add DNS entries for contoso.com and www.contoso.com, you must be able to configure the DNS settings for the contoso.comroot domain.

    Note

    If you don’t have an existing domain name, consider purchasing a domain using the Azure portal.

Prepare the app

To map a custom DNS name to a web app, the web app’s App Service plan must be a paid tier (SharedBasicStandard, or Premium). In this step, you make sure that the App Service app is in the supported pricing tier.

Note

App Service Free and Shared (preview) hosting plans are base tiers that run on the same Azure VM as other App Service apps. Some apps may belong to other customers. These tiers are intended to be used only for development and testing purposes.

Sign in to Azure

Open the Azure portal and sign in with your Azure account.

From the left menu, select App Services, and then select the name of the app.

Portal navigation to Azure app

You see the management page of the App Service app.

Check the pricing tier

In the left navigation of the app page, scroll to the Settings section and select Scale up (App Service plan).

Scale-up menu

The app’s current tier is highlighted by a blue border. Check to make sure that the app is not in the F1 tier. Custom DNS is not supported in the F1tier.

Check pricing tier

If the App Service plan is not in the F1 tier, close the Scale up page and skip to Map a CNAME record.

Scale up the App Service plan

Select any of the non-free tiers (D1B1B2B3, or any tier in the Production category). For additional options, click See additional options.

Click Apply.

Check pricing tier

When you see the following notification, the scale operation is complete.

Scale operation confirmation

Map your domain

You can use either a CNAME record or an A record to map a custom DNS name to App Service. Follow the respective steps:

Note

You should use CNAME records for all custom DNS names except root domains (for example, contoso.com). For root domains, use A records.

Map a CNAME record

In the tutorial example, you add a CNAME record for the www subdomain (for example, www.contoso.com).

Access DNS records with domain provider

Note

You can use Azure DNS to configure a custom DNS name for your Azure Web Apps. For more information, see Use Azure DNS to provide custom domain settings for an Azure service.

Sign in to the website of your domain provider.

Find the page for managing DNS records. Every domain provider has its own DNS records interface, so consult the provider’s documentation. Look for areas of the site labeled Domain NameDNS, or Name Server Management.

Often, you can find the DNS records page by viewing your account information, and then looking for a link such as My domains. Go to that page and then look for a link that is named something like Zone fileDNS Records, or Advanced configuration.

The following screenshot is an example of a DNS records page:

Example DNS records page

In the example screenshot, you select Add to create a record. Some providers have different links to add different record types. Again, consult the provider’s documentation.

Note

For certain providers, such as GoDaddy, changes to DNS records don’t become effective until you select a separate Save Changes link.

Create the CNAME record

Add a CNAME record to map a subdomain to the app’s default hostname (<app_name>.azurewebsites.net, where <app_name> is the name of your app).

For the www.contoso.com domain example, add a CNAME record that maps the name www to <app_name>.azurewebsites.net.

After you add the CNAME, the DNS records page looks like the following example:

Portal navigation to Azure app

Enable the CNAME record mapping in Azure

In the left navigation of the app page in the Azure portal, select Custom domains.

Custom domain menu

In the Custom domains page of the app, add the fully qualified custom DNS name (www.contoso.com) to the list.

Select the + icon next to Add hostname.

Add host name

Type the fully qualified domain name that you added a CNAME record for, such as www.contoso.com.

Select Validate.

The Add hostname page is shown.

Make sure that Hostname record type is set to CNAME (www.example.com or any subdomain).

Select Add hostname.

Add DNS name to the app

It might take some time for the new hostname to be reflected in the app’s Custom domains page. Try refreshing the browser to update the data.

CNAME record added

If you missed a step or made a typo somewhere earlier, you see a verification error at the bottom of the page.

Verification error

Map an A record

In the tutorial example, you add an A record for the root domain (for example, contoso.com).

Copy the app’s IP address

To map an A record, you need the app’s external IP address. You can find this IP address in the app’s Custom domains page in the Azure portal.

In the left navigation of the app page in the Azure portal, select Custom domains.

Custom domain menu

In the Custom domains page, copy the app’s IP address.

Portal navigation to Azure app

Access DNS records with domain provider

Note

You can use Azure DNS to configure a custom DNS name for your Azure Web Apps. For more information, see Use Azure DNS to provide custom domain settings for an Azure service.

Sign in to the website of your domain provider.

Find the page for managing DNS records. Every domain provider has its own DNS records interface, so consult the provider’s documentation. Look for areas of the site labeled Domain NameDNS, or Name Server Management.

Often, you can find the DNS records page by viewing your account information, and then looking for a link such as My domains. Go to that page and then look for a link that is named something like Zone fileDNS Records, or Advanced configuration.

The following screenshot is an example of a DNS records page:

Example DNS records page

In the example screenshot, you select Add to create a record. Some providers have different links to add different record types. Again, consult the provider’s documentation.

Note

For certain providers, such as GoDaddy, changes to DNS records don’t become effective until you select a separate Save Changes link.

Create the A record

To map an A record to an app, App Service requires two DNS records:

  • An A record to map to the app’s IP address.
  • TXT record to map to the app’s default hostname <app_name>.azurewebsites.net. App Service uses this record only at configuration time, to verify that you own the custom domain. After your custom domain is validated and configured in App Service, you can delete this TXT record.

For the contoso.com domain example, create the A and TXT records according to the following table (@ typically represents the root domain).

Record type Host Value
A @ IP address from Copy the app’s IP address
TXT @ <app_name>.azurewebsites.net

When the records are added, the DNS records page looks like the following example:

DNS records page

Enable the A record mapping in the app

Back in the app’s Custom domains page in the Azure portal, add the fully qualified custom DNS name (for example, contoso.com) to the list.

Select the + icon next to Add hostname.

Add host name

Type the fully qualified domain name that you configured the A record for, such as contoso.com.

Select Validate.

The Add hostname page is shown.

Make sure that Hostname record type is set to A record (example.com).

Select Add hostname.

Add DNS name to the app

It might take some time for the new hostname to be reflected in the app’s Custom domains page. Try refreshing the browser to update the data.

A record added

If you missed a step or made a typo somewhere earlier, you see a verification error at the bottom of the page.

Verification error

Map a wildcard domain

In the tutorial example, you map a wildcard DNS name (for example, *.contoso.com) to the App Service app by adding a CNAME record.

Access DNS records with domain provider

Note

You can use Azure DNS to configure a custom DNS name for your Azure Web Apps. For more information, see Use Azure DNS to provide custom domain settings for an Azure service.

Sign in to the website of your domain provider.

Find the page for managing DNS records. Every domain provider has its own DNS records interface, so consult the provider’s documentation. Look for areas of the site labeled Domain NameDNS, or Name Server Management.

Often, you can find the DNS records page by viewing your account information, and then looking for a link such as My domains. Go to that page and then look for a link that is named something like Zone fileDNS Records, or Advanced configuration.

The following screenshot is an example of a DNS records page:

Example DNS records page

In the example screenshot, you select Add to create a record. Some providers have different links to add different record types. Again, consult the provider’s documentation.

Note

For certain providers, such as GoDaddy, changes to DNS records don’t become effective until you select a separate Save Changes link.

Create the CNAME record

Add a CNAME record to map a wildcard name to the app’s default hostname (<app_name>.azurewebsites.net).

For the *.contoso.com domain example, the CNAME record will map the name * to <app_name>.azurewebsites.net.

When the CNAME is added, the DNS records page looks like the following example:

Portal navigation to Azure app

Enable the CNAME record mapping in the app

You can now add any subdomain that matches the wildcard name to the app (for example, sub1.contoso.com and sub2.contoso.com match *.contoso.com).

In the left navigation of the app page in the Azure portal, select Custom domains.

Custom domain menu

Select the + icon next to Add hostname.

Add host name

Type a fully qualified domain name that matches the wildcard domain (for example, sub1.contoso.com), and then select Validate.

The Add hostname button is activated.

Make sure that Hostname record type is set to CNAME record (www.example.com or any subdomain).

Select Add hostname.

Add DNS name to the app

It might take some time for the new hostname to be reflected in the app’s Custom domains page. Try refreshing the browser to update the data.

Select the + icon again to add another hostname that matches the wildcard domain. For example, add sub2.contoso.com.

CNAME record added

Edit the web.config using Kudu Console

There are two ways to access Kudu:

  1. Simply modify your website URL and by adding scm to it. If you site is http://mysite.azurewebsites.net/, then the root URL of the Kudu service is https://mysite.scm.azurewebsites.net/. Note the added scm token.
  2. Using the Azure Portal. First Navigate to your Web App, Select Tools -> Kudu -> Go:

launch Kudu from azure portal

How to View, Add, Edit, and Remove files in Azure Web App using Kudu

Finally this post was about how you actually view, edit, add, and remove files from the Web App. Once you have your Kudu service Dashboard open you will see some basic information and links for more complex tasks:

Kudu home page

View Current Files

View current files in your application by Clicking on Debug Console -> CMD:

Kudu view files

Once you are viewing the folder structure you can get to your application home directory by clicking the site folder:

Kudu site folder

 

Edit Files

To edit a file click the pencil icon:

launch Kudu from azure portal

Locate the web.config (or create it) and change the text to the following:

Replace <<appdomain>> with the domain you wish to redirect to.