This is Part 2 of a three-part series on running a real camera fleet on Azure IoT Operations (AIO):

In Part 1 the broker was just “a trusted MQTT bus” - Eclipse Mosquitto by default. This post replaces it with the Azure IoT Operations MQTT broker without touching a single line of camera or controller logic, and gets stronger security for free in the process. The whole point of AIO here is that it gives you an enterprise MQTT broker that runs on Kubernetes at the edge, with cloud-managed identity, authorization, and (in Part 3) data flows - while the application keeps speaking the exact same topics and payloads.

The seam that makes the swap possible

The control plane has three MQTT clients: the controller’s ingest service, the Class B agent, and the Class A site gateway. All three connect through one shared project, CameraNetwork.Mqtt, which exposes a MqttConnectionOptions record and a single extension method:

// Every client builds its options the same way:
var options = new MqttClientOptionsBuilder()
    .WithClientId(clientId)
    .Apply(connection)   // CameraNetwork.Mqtt resolves profile -> TLS / MQTT v5 / auth
    .Build();

MqttConnectionOptions carries a profile, an auth mode, and the TLS and credential details. There are two profiles:

  • Mosquitto (the default) - plain TCP on 1883, username/password, MQTT 3.1.1. Nothing in Part 1 changes.
  • AzureIotOperations - turns on TLS and MQTT v5 automatically, with Auth=X509 for clients outside the cluster or Auth=Sat (Kubernetes service-account token) for in-cluster components.

Because the profile resolves the transport details, swapping brokers is a configuration change, not a code change. The MQTT library already does v5, TLS, and X.509 client certificates - there is no new client dependency to swap a fleet onto AIO.

The broker seam: the controller ingest, izon-agent and site-gateway all build their MQTT options through CameraNetwork.Mqtt's .Apply(connection); the Mosquitto profile selects plain TCP on 1883, the AzureIotOperations profile selects the AIO broker on 8883 with TLS, X.509 and MQTT v5

Standing up Azure IoT Operations

AIO runs on an Azure Arc-enabled Kubernetes cluster. For a sandbox that is a single-node k3s box; in production it is whatever Arc-enabled cluster you run at the edge. The bring-up is a sequence of az commands (wrapped in a setup script in the repo), and it is deliberately a parallel sandbox - it never touches the existing Docker Compose or TrueNAS deployment, which keep using Mosquitto.

# Arc-connect the cluster and enable the features AIO needs
az connectedk8s connect --name cameranetwork-k3s --resource-group cameranetwork-aio --location eastus
az connectedk8s enable-features --name cameranetwork-k3s --resource-group cameranetwork-aio \
  --custom-locations-oid "$CL_OID" --features cluster-connect custom-locations

# A storage account + schema registry are required before 'az iot ops create'
az iot ops schema registry create --name cameranetwork-sr --resource-group cameranetwork-aio \
  --registry-namespace cameranetwork-sr-ns --sa-resource-id "$SA_ID"

# Initialize and create the AIO instance (this is the broker, dataflows, and the rest)
az iot ops init   --cluster cameranetwork-k3s --resource-group cameranetwork-aio
az iot ops create --cluster cameranetwork-k3s --resource-group cameranetwork-aio \
  --name cameranetwork-aio --sr-resource-id "$SR_ID"

When this finishes you have an AIO instance with an MQTT broker. AIO installs a default in-cluster listener (aio-broker:18883, TLS + service-account-token auth) for its own components. We leave that one alone and add a second listener for the cameras, which live outside the cluster.

A listener for cameras outside the cluster

Cameras connect from remote properties over the VPN, so they need a listener exposed off the cluster. On k3s, a LoadBalancer service picks up the node IP out of the box. The listener terminates TLS and references the X.509 authentication and authorization policies we define next.

apiVersion: mqttbroker.iotoperations.azure.com/v1
kind: BrokerListener
metadata:
  name: camera-external
  namespace: azure-iot-operations
spec:
  brokerRef: default
  serviceType: LoadBalancer
  serviceName: aio-broker-external   # do not clash with the default 'aio-broker' service
  ports:
    - port: 8883
      protocol: Mqtt
      authenticationRef: camera-x509-authn
      authorizationRef: camera-authz
      tls:
        mode: Automatic
        certManagerCertificateSpec:
          issuerRef:
            name: azure-iot-operations-aio-certificate-issuer   # the issuer AIO installs by default
            kind: ClusterIssuer
            group: cert-manager.io

tls.mode: Automatic lets cert-manager mint the listener’s server certificate from AIO’s default cluster issuer. That is fine for a sandbox; to tighten it you add the external IP or DNS name as a SAN so clients can do strict server-cert validation instead of trusting the CA directly.

Per-camera identity with X.509

This is where AIO earns its keep over plain Mosquitto. Each camera presents a client certificate, and the broker validates it against a CA we control. The repo’s cert helper generates a CA and a per-camera client cert whose common name equals the MQTT client id:

./make-camera-cert.sh izon-remote1-garage-east     # Class B agent
./make-camera-cert.sh site-gateway-remote2         # Class A gateway

The CA certificate is imported into the cluster as a ConfigMap, and a BrokerAuthentication resource trusts it. The clever bit is the root-subject mapping: every certificate signed by our CA inherits the attribute role: camera, so the entire fleet shares one identity policy with no per-device wiring.

apiVersion: mqttbroker.iotoperations.azure.com/v1
kind: BrokerAuthentication
metadata:
  name: camera-x509-authn
  namespace: azure-iot-operations
spec:
  authenticationMethods:
    - method: X509
      x509Settings:
        trustedClientCaCert: camera-client-ca
        authorizationAttributes:
          cameras:
            subject: CN = CameraNetwork Camera CA   # must match the CA subject exactly
            attributes:
              role: camera

One rule that reproduces the whole ACL

With every camera certificate carrying role: camera, authorization is a single allow rule. AIO authorization policies are allow-only - anything not granted is denied - so this one rule lets the whole fleet connect and use the cameras/# topic tree, which is exactly the wire contract from Part 1.

apiVersion: mqttbroker.iotoperations.azure.com/v1
kind: BrokerAuthorization
metadata:
  name: camera-authz
  namespace: azure-iot-operations
spec:
  authorizationPolicies:
    cache: Enabled
    rules:
      - principals:
          attributes:
            - role: camera
        brokerResources:
          - method: Connect
          - method: Publish
            topics: [ "cameras/#" ]
          - method: Subscribe
            topics: [ "cameras/#" ]

This is the AIO equivalent of the Mosquitto ACL. On Mosquitto you hand every camera a username/password and an ACL file entry; here you hand every camera a certificate from one CA and write one attribute-based rule. The trust anchor moves from a shared secret to a certificate authority, which is a real upgrade: revoking a camera is a CRL or re-issue operation, not a password edit pushed to a config file.

Per-camera X.509 identity: each camera client certificate is signed by the Camera CA, which BrokerAuthentication trusts; a root-subject mapping stamps every certificate with the attribute role=camera, which BrokerAuthorization allows to Connect, Publish and Subscribe on the cameras topic tree at the external listener on 8883

If you later want per-camera isolation - a camera that can only touch its own channels - you give each certificate site and camera attributes in the authentication resource, then scope the authorization topics by token substitution, e.g. cameras/{principal.attributes.site}/{principal.attributes.camera}/+. That mirrors a per-camera Mosquitto ACL, but it is optional; the single fleet-wide rule is the starting point.

Pointing a component at AIO

Nothing about the swap is code. You flip the profile and point at the external listener. A Class B agent’s settings:

{
  "Agent": {
    "SiteId": "remote1",
    "CameraId": "garage-east",
    "MqttProfile": "AzureIotOperations",
    "MqttHost": "<EXTERNAL-IP>",
    "MqttPort": 8883,
    "MqttAuth": "X509",
    "MqttClientCertPfxPath": "/certs/izon-remote1-garage-east.pfx",
    "MqttCaCertPath": "/certs/camera-ca.crt",
    "MqttAllowUntrustedCertificates": true
  }
}

The controller is the same idea with its own config prefix (environment variables shown):

Mqtt__Profile=AzureIotOperations
Mqtt__Host=<EXTERNAL-IP>
Mqtt__Port=8883
Mqtt__Auth=X509
Mqtt__ClientCertPfxPath=/certs/camera-controller.pfx
Mqtt__CaCertPath=/certs/camera-ca.crt

MqttProfile=AzureIotOperations turns on TLS and MQTT v5 automatically. MqttAllowUntrustedCertificates=true skips server-cert validation and is a sandbox-only shortcut - drop it once the listener’s server cert carries the external IP as a SAN and rely on the CA file instead. The controller can also run inside the cluster with Auth=Sat and Host=aio-broker, using the default internal listener.

Verify the fleet landed on the broker

Check the broker resources, then watch a real camera connect over TLS with its client certificate:

# Broker resources are healthy
kubectl get brokerlistener,brokerauthentication,brokerauthorization -n azure-iot-operations
az iot ops check

# Subscribe from your workstation with the CA + a camera cert/key
mosquitto_sub -h <EXTERNAL-IP> -p 8883 -V mqttv5 -t 'cameras/#' -v \
  --cafile certs/camera-ca.crt \
  --cert certs/izon-remote1-garage-east.crt \
  --key certs/izon-remote1-garage-east.key

Start the agent with the AIO config and you see its retained availability and inventory, then periodic status. Point the controller at the same listener and the camera appears on the dashboard exactly as it did on Mosquitto. Kill the agent and its availability flips to offline through the Last-Will - the retained-message and Last-Will-Testament behavior carries across brokers, which is the real test that the swap is transparent.

Why this matters

The Mosquitto deployment trusts the private VPN and VLAN: the broker is never exposed to untrusted networks, hardening is per-camera username/password plus ACLs, and the firewall makes port 1883 reachable only from the controller. That is a perfectly good model on a trusted segment.

Moving to the AIO broker upgrades the trust model without changing the application:

  • Identity is a certificate from a CA you control, not a shared secret.
  • TLS is on by the profile, so cameras can cross less-trusted segments.
  • Authorization is attribute-based and cloud-managed, with one rule for the fleet and an obvious path to per-camera scoping.
  • The broker is now a managed Kubernetes workload with health you can query (az iot ops check), not a single container.

And critically, the message contract from Part 1 is untouched. That is what sets up Part 3: once the fleet is publishing to the AIO broker, AIO can route that traffic to the cloud and onboard new kinds of cameras as assets - again with no application change.

Continue to Part 3: data flows, connectors, and the cloud.

References