By this point in the series the demos exist as stories: routing, private RAG, camera operations, Azure governance, failure behavior, and accelerator backends. What the series still needed was the machine those stories run on. So this post is the engineering brief for building the presentation repo, written so another engineer - or an agent - can implement it without rediscovering the story first. The standard throughout is that the system should be boring to run. All of the excitement belongs on stage.
Why the Repo Has to Be Boring
A conference demo that needs thirty manual steps is a liability. Miss one step in a hotel room the night before, and talk day turns into live-debugging in front of people who came for architecture. Determinism is not a preference here; it is the design constraint. The repo needs deterministic setup, seed data, reset behavior, smoke tests, local mode, edge mode, and clear health endpoints.
The target is not “works on my machine after I remember the sequence.” The target is:
make demo-laptop
make demo-edge
make demo-seed
make demo-reset
make demo-smoke-test
make infra-plan
make infra-apply
make teardown
The first five targets drive the demo itself; the last three manage the Azure-governed mode’s resources. If those commands exist and mean something, the architecture can survive rehearsals, travel, hotel Wi-Fi, and last-minute hardware failures. I have watched enough talks die in the third minute to want the repo doing the remembering instead of me.
Repo Layout and Shared Contracts
The repo is organized around service ownership and demo modes:
ai-on-the-edge/
docs/
architecture/
decision-records/
diagrams/
session-material/
src/
AiOnTheEdge.Gateway/
AiOnTheEdge.Routing/
AiOnTheEdge.KnowledgeAssistant/
AiOnTheEdge.OperationsAssistant/
AiOnTheEdge.ControlDashboard/
AiOnTheEdge.Telemetry/
AiOnTheEdge.DemoScenarios/
infra/
azure/
kubernetes/
arc/
local/
dashboards/
policies/
samples/
documents/
camera-events/
prompts/
evaluations/
scripts/
demo/
setup/
test/
talks/
An honest caveat about that tree: it and the contracts below describe my private reference implementation. The repo is not public, so treat the layout as the specification an implementation should satisfy rather than a checkout you can clone.
Every service exposes the same operational endpoints as a floor:
/healthz
/readyz
/metrics
/admin/demo/reset
/admin/demo/seed
/admin/demo/faults
Individual services add to that list rather than replacing it. The Knowledge Assistant carries /admin/reindex on top of the six; the Operations Assistant carries its replay controls. The six above are what a health check, a reset script, or a smoke test can assume without knowing which service it is talking to.
And every AI request emits the same route decision event shape (field values below are illustrative, not measurements):
{
"timestamp": "2026-06-30T12:00:00Z",
"scenario": "private-rag",
"classification": "restricted",
"requestedModel": "local-chat",
"selectedBackend": "foundry-local",
"decision": "Allowed",
"policy": "LocalOnly",
"fallbackUsed": false,
"promptBodyLogged": false,
"latencyMs": 842,
"promptTokens": 640,
"completionTokens": 122,
"reason": "Restricted content must stay on edge."
}
requestedModel records the concrete model the router resolved the requested family to, which is why it reads local-chat rather than chat. That single event is the contract between the gateway, dashboard, metrics, logs, KQL queries, and the talk narrative. When something looks wrong on stage, this is the artifact that explains why.
Three Ways to Run It
The implementation supports three modes. Laptop mode is the reliable conference fallback: .NET app, local or mock model runtime, local vector store, simulated camera events. Edge mode runs Kubernetes or k3s with the gateway, assistants, dashboard, MQTT, simulator, and observability. Azure-governed mode layers Arc, Azure Monitor, Managed Prometheus, Grafana, Key Vault, GitOps, optional Azure IoT Operations, and optional Azure Local on top.
Laptop mode is the default live path. Edge mode proves the system is deployable. Azure-governed mode proves the operating model.
Build in Demo Order
Milestones go in the order they appear on stage, so every stretch of build time produces something demonstrable:
| Unlocks | Build |
|---|---|
| 1. Gateway demo | Gateway, mock backends, policies, dashboard, route events, make demo-laptop. |
| 2. Private RAG | Document ingestion, chunking, embeddings, vector store, citations, RAG evaluation. |
| 3. Operations assistant | Camera replay, MQTT ingest, event normalization, incidents, operator summaries. |
| 4. Azure operations | Infra, Kubernetes manifests, dashboards, KQL, GitOps, Key Vault, policy examples. |
| 5. Failure lab | Fault injection, cloud-blocked mode, smoke tests, reset controls. |
| 6. Accelerator lane | Tenstorrent adapter hardening, capability discovery, health checks, mock accelerator. |
Do not start with the portal. Start with the deterministic local path, then add cloud governance around it.
How the 45 Minutes Actually Run
A 45-minute talk should not run every possible path live. The recommended flow:
| Time | Segment | Demo |
|---|---|---|
| 0:00-3:00 | Problem | Local AI is useful, unmanaged edge AI is model sprawl. |
| 3:00-7:00 | Architecture | Gateway, router, policy engine, Azure operations layer. |
| 7:00-15:00 | Live Demo 1 | One app, many model targets. |
| 15:00-24:00 | Live Demo 2 | Private RAG that cannot leave the edge. |
| 24:00-34:00 | Live Demo 3 | Camera fleet operations assistant. |
| 34:00-40:00 | Demo 4 | Azure governance and observability dashboard. |
| 40:00-43:00 | Failure cut-in | Disable backend, deny cloud fallback, show telemetry. |
| 43:00-45:00 | Close | Cloud-governed, locally executed. |
Optional cut-ins are Foundry Local on Azure Local, Tenstorrent hardware, AIO data flow to Event Hubs, and disconnected mode. They should be additive, not dependencies; the talk has to work with every one of them missing, which is also why nothing cloud-side gets installed live during the session.
What This Post Locks In
The blog series now defines the acceptance criteria and audience moments, the private cloud post defines the physical environment, and the camera posts define the IoT workload; an implementation should treat all of that as requirement inputs. This post adds the implementation contract:
- The repo layout.
- The make targets.
- The common service endpoints.
- The route decision event.
- The build order.
- The presentation flow.
- The standard for deterministic demo behavior.
None of that is glamorous. It is the part that decides whether the demos behave the same way twice.
The Pile of Disconnected Samples
There is a specific way this repo could fail while every individual piece works: seven services’ worth of clever code that together demonstrate nothing. Every service should contribute to the same route decision, metrics, dashboard, and reset story.
If a feature does not help the session or make the system more repeatable, it can wait. It will still be there after the talk.
Ready to Rehearse
The system is ready when the make targets mean what they say. make demo-laptop brings up the primary path with no cloud involved, make demo-edge deploys the same thing onto local Kubernetes, and make demo-seed, make demo-reset, and make demo-smoke-test respectively create the documents, camera events, backend config and policies, put all of it back to a known state, and prove that routing, RAG, incidents, faults, and accelerator fallback still behave. On the Azure side, make infra-plan and make infra-apply provision the governed mode repeatably and make teardown takes it back down without leaving anything billable behind.
Two smaller checks matter more than they look. Every service answers on health, readiness, metrics, seed, reset, and faults, so nothing in the demo needs a service-specific runbook. And the dashboard shows the current state before I say a word, which is how I find out whether the system is ready without narrating a diagnostic to the room. When all of that passes, the night before the talk is for sleeping, not for shell scripts.
Related Posts
This is the build companion for the whole AI on the Edge series, starting from the gateway in One App, Many Places to Run AI and the failure harness in When the Edge Has to Stand Alone. The final post, the AI on the Edge reference architecture, turns the system into an evergreen reference.