Buildout Overview

The basement is now doing two jobs at once: studio and server host. The studio is intentionally simple, just a green screen wall and a demo table, because the focus is repeatable technical demonstrations across multiple industry verticals. Right behind that space is the local compute footprint that powers the demos.

This buildout now includes two custom Tenstorrent server paths mixed into my existing NVIDIA-enabled and general-purpose server stack. The point is not to build isolated one-off demos, but to create a reusable local platform that can be switched between demo scenarios with minimal rework.

The broader architecture is hybrid by design. I want the same local inference flow to run with multiple cloud providers, plus a purely local mode when needed. That gives us a practical way to show customers and internal teams what edge-first compute can do, and where cloud orchestration adds value.

Terminology Rule

• Private Cloud means self-hosted infrastructure in my server room.
• Local means local-only execution paths and equivalent services running in Kubernetes.
• This write-up consistently uses Private Cloud or Local terminology based on context.

Private Cloud Topology

Azure Integration View

Control Plane Overlay (Azure + Private Cloud)

AWS Integration View

Local-Only Implementation (Specific Self-Hosted Tools)

In this local view, each stage is concrete and self-hosted, but still interchangeable by contract. The handoff boundaries are RTSP ingest, MQTT frame topics, event stream topics, and packaged artifact output.

Physical Lab and Studio Layout

The studio side is optimized for fast context switching. I can record walkthroughs, run live demos, and pivot from one vertical scenario to another without rebuilding the room. The server side is optimized for shared hardware utilization across those same scenarios.

This setup makes it easier to:

• Reuse the same edge hardware for multiple business demos.
• Keep model and telemetry pipelines consistent across environments.
• Demonstrate cloud-assisted operations without requiring cloud-only inference.
• Keep local fallback paths available when connectivity is constrained.

Current Lab Inventory

Current internal lab components include:

• Windows RTSP stream host
• Wormhole server
• Blackhole server path: staged and not fully onboarded yet
• Two active web cams for a high shot and a low shot
• TrueNAS storage box with two Tesla P400 GPUs running Ollama and other workloads
• Proxmox hosts for general private cloud workloads

The practical goal is to run a mixed accelerator lab where workload placement can be tuned by use case, latency target, and cost profile.

Azure Path: IoT Operations, Arc, and Edge Feedback Loops

On Azure, the control pattern is centered around Azure IoT Operations on Azure Arc-enabled Kubernetes. In the demo repository, this aligns with the Azure/Demo/Shared/AzureInternetOfThingsOperations assets and the shared EdgeInference service.

The local flow is:

1. Cameras publish RTSP to the Windows RTSP host.
2. The AIO RTSP Adapter ingests RTSP and publishes frames to the local MQTT frame topic.
3. Edge inference services consume those frame topics and publish detections, traffic, and enriched messages.
4. AIO Data Flows process, normalize, and extract delta events.
5. Data Flows push upstream to Fabric RTI and call a Foundry endpoint for cloud verification.
6. Results flow into cloud analytics and operations dashboards, with feedback updates pushed back to edge.

In repository terms, that includes topic patterns such as:

• tt/edge/{site}/{camera_id}/detections
• tt/cloud/{site}/{device_id}/verify
• tt/cloud/{site}/{device_id}/verify-result

Operationally, Arc gives me a consistent management surface for local Kubernetes resources and policy. I am also treating GitOps with Flux on Arc-enabled Kubernetes as the default deployment and configuration strategy for repeatability.

One reason this fits the basement buildout well is that Azure IoT Operations is designed as a unified edge data plane with an industrial MQTT broker and supports routing/normalization before cloud fan-out. That maps directly to how I want to keep high-volume inference local while still enabling cloud-side verification, model lifecycle workflows, and cross-site analytics.

For the Fabric path, I am modeling Data Flows publishing directly to Fabric Eventstream through the documented Fabric endpoint configuration (no required Event Hub bridge in this path).

AWS Path: Greengrass, IoT Core, SiteWise, Systems Manager, and Bedrock

On AWS, the equivalent pattern uses:

• AWS IoT Greengrass for edge runtime orchestration.
• AWS IoT Core for secure bi-directional cloud messaging and device state.
• AWS IoT SiteWise for industrial telemetry modeling, transforms, and monitoring.
• AWS Systems Manager for hybrid fleet operations, patching, and command execution.
• Amazon Bedrock as part of the cloud-side model specialization and distillation path.

The AWS side in the demo repository is organized to mirror Azure demo shape where possible, including shared vertical scenarios and analytics assets. The intent is to keep edge behavior portable while changing only cloud control-plane integrations.

At a high level:

1. Local inference continues at the edge.
2. Edge messaging bridges into AWS IoT Core patterns.
3. Industrial telemetry and KPI modeling feed SiteWise analytics.
4. Operations and lifecycle tasks route through Systems Manager.
5. Distilled cloud-side model workflows can feed edge deployment artifacts.

This gives us realistic AWS parity for customer conversations where cloud preference is fixed but the edge architecture should stay consistent.

Cross-Provider Pattern

The architecture pattern stays the same even when control planes differ:

Layer	Azure	AWS
Edge inference runtime	Tenstorrent edge service on local K8s	Tenstorrent workloads under Greengrass-managed edge runtime
Edge messaging	IoT Operations MQTT broker	IoT Core and Greengrass local messaging patterns
Fleet and policy	Arc-enabled infrastructure and GitOps	Systems Manager + IoT fleet operations
Industrial analytics	Event flow to cloud analytics services	SiteWise asset model and telemetry analytics
Model lifecycle	Cloud verification + model workflows	Bedrock-assisted distillation workflows

This is the core reason for the buildout: one local edge core, multiple cloud orchestration options, and a purely local fallback.

First Milestone

The first milestone for this personal buildout is to get Azure IoT Operations fully wired to local Tenstorrent inference in a closed loop:

1. Ingest local RTSP streams into the edge inference service.
2. Run local inference on the Tenstorrent host.
3. Publish structured detections into IoT Operations topics.
4. Forward selected events or sampled frames for cloud verification.
5. Return verification and analytics outcomes to operational dashboards.
6. Push updated thresholds and model-management decisions back to the edge.

This gives a concrete demonstration of edge inference plus cloud feedback rather than edge-only or cloud-only narratives.

Next Steps

1. Complete Blackhole onboarding and benchmark against current Wormhole and NVIDIA paths.
2. Harden deployment automation across Azure and AWS for faster scenario switching.
3. Expand vertical demo packs so the same local hardware can represent more business contexts.
4. Add stronger runbook-level operational checks for edge health, topic flow integrity, and model rollout safety.

References

• Azure IoT Operations overview
• Configure Azure IoT Operations Fabric endpoint
• Azure Arc overview
• Azure Arc-enabled Kubernetes overview
• GitOps with Flux v2 on Azure Arc-enabled Kubernetes
• AWS IoT Greengrass overview
• AWS IoT Core overview
• AWS IoT SiteWise overview
• AWS Systems Manager overview
• Amazon Bedrock overview
• Tenstorrent documentation home
• Tenstorrent TT-Forge documentation
• Tenstorrent Wormhole hardware
• Tenstorrent Blackhole hardware