Process Incoming IoT Data in Real Time

In this IT Pro Challenge virtual lab, you will get hands-on experience using Azure’s IoT Hub to collect IoT device telemetry and create Stream Analytics to process and analyze the data. These skills will allow an IoT/OT Administrator or Security Analyst to understand how to manage, monitor effectively, and analyze your IoT environment in Azure.

1 hour
Join over 3 million cybersecurity professionals advancing their career
Sign up with
Required fields are marked with an *

Already have an account? Sign In »


Lab Overview:

This hands-on lab provides an Azure administrator with an understanding of how to configure your environment to process incoming IoT telemetry data in real-time. (Note: This is an advanced challenge and part of a series. You may want to attempt the Guided Challenges “Process IoT Data with Stream Analytics” and “Implement IoT Hub Consumer Groups and Endpoints” before attempting this advanced challenge.) You will learn how to consume data, run analytics against it, and distribute the output to consumer groups. These skills are essential for IoT/OT Administrators or Security Analyst.

Understand the scenario

You are a system administrator for a company that is rolling out an Internet of Things (IoT) data solution. You need to provide IoT telemetry data processing capabilities for the company’s data streams. You will start by creating a secondary endpoint and consumer groups. You will then provision a Stream Analytics job that routes data to a Storage Account Table. Finally, you will verify your results through a test web site. To complete these tasks, you will use an Azure resource group that contains a preconfigured web app, an IoT Hub, and a storage account.

Configure a web app to use an Azure IoT Hub:

The Azure lab environment is provisioned with an IoT Hub, a Web App, and a Storage Account. For the first part of this lab, you will configure your web app with connection strings (i.e., an IoT hub permission connection string primary key and a string for configuring an endpoint for events). Then you will provision the IoT devices that will be monitored in your environment. You will simulate collecting events from a building, a vehicle, and a piece of equipment.

Create an endpoint and consumer groups:

For this task, you will configure a blob endpoint and a route for your message traffic to go to the blob endpoint. Then you will provision two consumer groups that will both receive the IoT telemetry log data. Then you will test that data by simulating sending events from the three IoT devices you’ve configured. The device emulation within the web app is based on the same IoT code that would be used on an actual IoT device, and the emulation interacts with the Azure IoT hub in the same way as an IoT device would.

Provision a Stream Analytics job:

Microsoft Azure provides Stream Analytics, a real-time event processing engine for managing, monitoring, and performing use-case driven analytics on streaming data from your IoT (and other types of) devices. For this task, you will create a Stream Analytics job, configure a stream input, and add the type of output you desire (in this case, datastore table storage).

Define the Stream Analytics query:

For this task, you will configure a test query to make sure that the data is available, then you will refine the query to output the data in a useful format for monitoring. This lesson will provide exposure to the query language used within Stream Analytics, which is SQL-like.

Lab Summary Conclusion:

In this hands-on virtual lab, you will learn how to configure an Azure IoT Hub and Stream Analytics to manage the input and output of IoT device telemetry data. These skills will allow you to understand how Azure can be used to manage, monitor effectively, and analyze your IoT environment. This capability can enable you to perform analytics for issues that may arise in your environment, such as device maintenance requirements, security issues, misconfigurations, or other critical insights. These skills are essential for someone pursuing a career in the IoT/OT space and correlate to the skills needed for IT security operations. These skills are valuable to an Azure Administrator and an IT/IoT/OT NOC or SOC Analyst.

Other Challenges in this series

  • GUIDED CHALLENGE - Process IoT Data with Stream Analytics
  • GUIDED CHALLENGE - Implement IoT Hub Consumer Groups and Endpoints