| title | description | ms.service | services | author | ms.author | ms.topic | ms.date |
|---|---|---|---|---|---|---|---|
Process messages from your IoT assets and devices | An overview of message processing options in an Azure IoT solution including routing and enrichments. | azure-iot | iot | asergaz | sergaz | overview | 03/24/2025 |
This overview introduces the key concepts around processing messages sent from your assets and devices in a typical Azure IoT solution. Each section includes links to content that provides further detail and guidance.
The following diagram shows a high-level view of the components in a typical edge-based IoT solution. This article focuses on the message processing components of an edge-based IoT solution.
:::image type="content" source="media/iot-overview-message-processing/iot-edge-message-architecture.svg" alt-text="Diagram that shows the high-level edge-based IoT solution architecture highlighting message processing areas." border="false" lightbox="media/iot-overview-message-processing/iot-edge-message-architecture.svg":::
The following diagram shows a high-level view of the components in a typical cloud-based IoT solution. This article focuses on the message processing components of a cloud-based IoT solution.
:::image type="content" source="media/iot-overview-message-processing/iot-cloud-message-architecture.svg" alt-text="Diagram that shows the high-level cloud-based IoT solution architecture highlighting message processing areas." border="false" lightbox="media/iot-overview-message-processing/iot-cloud-message-architecture.svg":::
In Azure IoT, message processing refers to processes such as routing and enriching messages sent by assets and devices. These processes are used to control the flow of messages through the IoT solution and to add additional information to the messages.
To route messages from your assets to various endpoints, Azure IoT Operations uses data flows. The destination endpoints might be in the cloud or at the edge. The list of available destination endpoints includes:
| Endpoint type | Description |
|---|---|
| MQTT | For bi-directional messaging with MQTT brokers, including the one built-in to Azure IoT Operations and Event Grid. |
| Kafka | For bi-directional messaging with Kafka brokers, including Azure Event Hubs. |
| Data Lake | For uploading data to Azure Data Lake Gen2 storage accounts. |
| Microsoft Fabric OneLake | For uploading data to Microsoft Fabric OneLake lakehouses. |
| Azure Data Explorer | For uploading data to Azure Data Explorer databases. |
| Local storage | For sending data to a locally available persistent volume, optionally configurable with Azure Container Storage enabled by Azure Arc. |
The operations experience web UI provides a no-code environment for building and running your data flows.
For enhanced security in the data that is routed to your endpoints, cloud/edge synchronized secrets are used in data flow endpoints for authentication.
While data flows let you configure routing at the edge, you can also define routing in the cloud. If your data flow delivers messages to Azure Event Grid you can use its routing capabilities to determine where to send the messages.
To learn more, see Process and route data with data flows.
An Azure IoT hub provides a cloud entry point for the telemetry messages that your devices send. In a typical IoT solution, these messages are delivered to other downstream services for storage or analysis.
In IoT hub, you can configure routing to deliver telemetry messages to the destinations of your choice. Destinations include:
- Storage containers
- Service Bus queues
- Service Bus topics
- Event Hubs
Every IoT hub has a default destination called the built-in endpoint. Downstream services can connect to the built-in endpoint to receive messages from the IoT hub.
To learn more, see Use IoT Hub message routing to send device-to-cloud messages to different endpoints.
You can use queries to filter the messages sent to different destinations.
If you're using Azure IoT Central, you can use data export to send telemetry messages to other downstream services. Destinations include:
- Storage containers
- Service Bus queues
- Service Bus topics
- Event Hubs
- Azure Data Explorer
- Webhooks
An IoT Central data export configuration lets you filter the messages sent to a destination.
To learn more, see Export data from IoT Central.
IoT Hub has built-in integration with Azure Event Grid. An IoT hub can publish an event whenever it receives a telemetry message from a device. You can use Event Grid to route these events to other services.
To learn more, see React to IoT Hub events by using Event Grid to trigger actions and Compare message routing and Event Grid for IoT Hub.
Enrichments and transformations can be applied to data during the processing stage to perform various operations. These operations can include:
- Compute new properties: Based on existing properties in the message
- Rename properties: To standardize or clarify data
- Convert units: Convert values to different units of measurement
- Standardize values: Scale property values to a user-defined range
- Contextualize data: Add reference data to messages for enrichment and driving insights
The schema registry stores schemas for messages coming from your assets. Data flows use these message schemas to decode messages from various formats so they can be processed by data flows.
The operations experience web UI provides a no-code environment for building and running the transformations in your data flows.
To learn more, see Enrich data by using data flows.
In Azure IoT Operations, you can deploy your own highly available edge applications to the Kubernetes cluster. The edge applications can interact with the built-in MQTT broker to:
- Use custom message processing logic on the MQTT messages.
- Build custom application logic to run at the edge.
- Run Edge AI models for real-time data processing and decision-making at the source of data generation, reducing latency and bandwidth usage.
To learn more, see Develop highly available applications for Azure IoT Operations MQTT broker.
To simplify downstream processing, you might want to add data to telemetry messages or modify their structure.
IoT Hub message enrichments let you add data to the messages sent by your devices. You can add:
- A static string
- The name of the IoT hub processing the message
- Information from the device twin
To learn more, see Message enrichments for device-to-cloud IoT Hub messages.
IoT Central has two options for transforming telemetry messages:
- Use mappings to transform complex device telemetry into structured data on ingress to IoT Central.
- Use transformations to manipulate the format and structure of the device data before you export it to a destination.
An Azure IoT Edge module can process telemetry from an attached sensor or device before it sends it to an IoT hub. For example, before it sends data to the cloud an IoT Edge module can:
You can use other cloud services to process messages from your assets and devices.
Data flow endpoints in Azure IoT Operations let you connect to cloud services to send and receive data from your assets. A data flow endpoint is the connection point for the data flow.
To learn more, see:
In IoT Hub and IoT Central, you can route messages to other services. For example, you can forward messages to Azure Stream Analytics to analyze and process large volumes of streaming data or to Azure Functions to run code in response to events. Stream Analytics is also available on the Azure IoT Edge runtime, enabling it to process data at the edge rather than in the cloud.
To learn more, see:
