Azure Event Hub

Video Activity
Join over 3 million cybersecurity professionals advancing their career
Sign up with
Required fields are marked with an *
or

Already have an account? Sign In »

Time
14 hours 28 minutes
Difficulty
Intermediate
CEU/CPE
15
Video Transcription
00:00
>> Hello, Cybrarians.
00:00
Welcome to Lesson 7.4 of Module 7 of this course
00:00
titled AZ-301: Microsoft Azure Architect Design.
00:00
Here are the learning objectives for this video.
00:00
We'll start out by introducing Azure Event Hub.
00:00
What exactly is this service? What does it do?
00:00
We'll then proceeds to cover
00:00
Azure Event Hub from an internal perspective,
00:00
how it does this service do
00:00
what it does, how does it function?
00:00
We'll cover some co-features of Azure Event Hub.
00:00
Then we'll do some comparison of
00:00
event message and services in Azure.
00:00
Let's get into this. Let's talk about Azure Event Hub.
00:00
According to Microsoft, Azure Event Hub is
00:00
a big data streaming platform
00:00
and event ingestion service.
00:00
It can receive and process
00:00
millions of events per seconds.
00:00
Data sent to an Event Hub can be
00:00
transformed and stored by using
00:00
any real-time analytics provider
00:00
or batching/storage adapters.
00:00
What does that mean?
00:00
That's the question that I'd like to ask.
00:00
What does that mean? Let's talk about Azure Event Hub.
00:00
Azure Event Hub is described by Microsoft as
00:00
a big data streaming platform
00:00
and an event ingestion service.
00:00
What exactly does that mean?
00:00
Here's what that means. In many cases,
00:00
when we work with data,
00:00
the data that will work which
00:00
does not come from a single source,
00:00
and most likely there are many sources that
00:00
produce data that would need to
00:00
be able to unlock insight.
00:00
Maybe data coming from IoT devices, from sensors,
00:00
from web and mobile applications,
00:00
datas coming from application telemetry,
00:00
metrics, and then many other sources.
00:00
This information usually needs to be
00:00
available for processing by other systems.
00:00
Usually, the other systems
00:00
>> that also needs to be able to receive
00:00
>> and process or analyze these data,
00:00
>> it's not just one,
00:00
there are multiple of them.
00:00
What Azure Event Hub allows us to do
00:00
>> is to be able to ingest data from a variety of sources
00:00
>> and reliably distributes that data
00:00
>> between multiple independent systems for processing,
00:00
>> for storage, and for more deep analysis.
00:00
Let's bring this on a little bit.
00:00
If we're working with Azure Event Hub,
00:00
the first thing that we need to create is
00:00
something called an Event Hub namespace.
00:00
After we create our Azure Event Hub namespace,
00:00
then we can create our Event Hub within the namespace.
00:00
Now, when we're creating our Event Hub,
00:00
we need to define something called partitions.
00:00
Now, the number of partitions that we specify is very
00:00
important because it's not something
00:00
that we can modify afterwards.
00:00
Once you've selected the number of partitions,
00:00
that's the number of partitions for that Event Hub.
00:00
What does the partition corresponds to?
00:00
The number of partitions in an Event Hub directly
00:00
>> relates to the number of concurrent readers
00:00
>> that you expect to have.
00:00
>> That is what that refers to,
00:00
the number of concurrent readers
00:00
>> that we intend to have.
00:00
>> For example, if I have multiple event producers
00:00
>> that are sending message either via HTTP
00:00
>> or AMQP into Azure Event Hub service,
00:00
>> and I have all these different consumer groups
00:00
>> which are the event receivers that will be streaming
00:00
>> this event into their platforms,
00:00
then I need to ensure that
00:00
>> I have the right number of partitions
00:00
>> for them to be able to consume
00:00
those events streaming concurrently.
00:00
Some features of Azure Event Hub.
00:00
Number 1, Azure Event Hub, is highly scalable.
00:00
It can scale up to millions of events per second.
00:00
That's not per minute, that's within a second.
00:00
It supports Apache Kafka
00:00
without the infrastructure management overhead.
00:00
Azure Event Hub provides a Kafka endpoints that can be
00:00
used by existing Kafka based applications.
00:00
That will be as an alternative to you running
00:00
your own Kafka cluster
00:00
>> and building out the infrastructure and managing that.
00:00
>> Essentially Azure Event Hub can be your platform
00:00
>> as a service managed Kafka cluster.
00:00
>> You don't have to deal with
00:00
>> the infrastructure for that.
00:00
>> Azure Event Hub support
00:00
Apache Kafka protocol 1.0 and later.
00:00
If you have existing Kafka applications that
00:00
supports this level of protocol and later,
00:00
you can essentially use Azure Event Hub
00:00
to replace your Kafka cluster.
00:00
It supports real-time and batch processing.
00:00
Its real-time use case is very interesting
00:00
>> because that can enable you
00:00
>> to stream that directly into a service
00:00
>> that does analytics like Azure Stream Analytics
00:00
>> and then you can unlock insights in real-time.
00:00
>> It supports data capture
00:00
to Azure Blob and Azure data lake.
00:00
As data is being streamed through this service,
00:00
you can decide to do something called a data capture
00:00
>> and you can capture that straight
00:00
>> into an Azure Blob storage or Azure data lake.
00:00
>> Then you can go back to review
00:00
>> what's been streamed through the service.
00:00
>> It has integration with Dynamics 365.
00:00
It has integration with Azure Stream Analytics
00:00
>> so we can unlock real-time analytics.
00:00
>> Now, if you're thinking to yourself, David,
00:00
a lot of these services
00:00
>> that you've described to us sound very similar.
00:00
>> They seem to have publish-subscribe model
00:00
>> where you have some creators on the left-hand side
00:00
>> and some consumers on the right-hand side.
00:00
For all these different services,
00:00
they look very similar to me.
00:00
What's the difference between them?
00:00
That's what the next two slides is about.
00:00
To understand the differences between them,
00:00
we first of all need to understand
00:00
the differences between events and the message.
00:00
When we're talking about event-driven architecture
00:00
versus message-driven architecture.
00:00
When we talk about an event,
00:00
an event is a lightweight notification
00:00
of a condition of a state change.
00:00
What that means is that
00:00
an event is not necessarily raw data.
00:00
A good example of that would be
00:00
>> if data or file or an object is going to be
00:00
>> dropped into a storage account, a Blob storage,
00:00
and I'm going to use an event
00:00
to trigger another execution
00:00
of another service after that,
00:00
now, that event data has
00:00
information about what happened,
00:00
but it doesn't have the file itself.
00:00
The file that was dropped
00:00
>> into that Blob storage account
00:00
>> is not what the event carries.
00:00
>> It only contains information to say,
00:00
a file was dropped.
00:00
What exactly is inside this file,
00:00
it does not have that information.
00:00
For example, an event notifies
00:00
consumers that a file was created,
00:00
it may have general information about
00:00
the file but it doesn't have the file itself.
00:00
Compare that to a message.
00:00
A message is raw data
00:00
that produced by a service
00:00
to be consumed or stored elsewhere.
00:00
A message can be the file itself,
00:00
the raw data itself
00:00
that's now been produced by a service,
00:00
not information about an event that occurred,
00:00
but the actual raw data itself.
00:00
Typically, when you talk about messages,
00:00
there's a relationship between
00:00
the message producer and the message consumer.
00:00
Where the message producer in some essence
00:00
>> as an expectation of what
00:00
>> the message consumer will do with that.
00:00
When we talk about events,
00:00
there's typically no relationship between that.
00:00
For example, if a file is stored into a Blob storage,
00:00
I'm going to use that file of an Azure function.
00:00
Blob storage doesn't have a relationship to say,
00:00
hey, I expect you to do this.
00:00
An event just happens and then that's pushed out
00:00
>> to another service that's interested in that event.
00:00
>> That being said, you can see the differences
00:00
between the services at the top there.
00:00
For Event Grid,
00:00
it supports reactive programming.
00:00
Event Hub is mainly
00:00
>> for big data pipeline and for time-series,
00:00
>> while Service Bus is
00:00
>> for high-value enterprise messaging.
00:00
>> When we talk about the types of data that they handle,
00:00
Event Grid handles event distribution,
00:00
which are discrete event.
00:00
While Event Hub handles
00:00
>> streaming of a time series event.
00:00
>> Things like log files versus
00:00
Event Grid that's just discrete event,
00:00
and Service Bus messages raw data.
00:00
When to use them, you use Event Grid
00:00
>> when you want to react to status changes.
00:00
>> You use the Event Hub
00:00
>> when you want to do telemetry
00:00
>> and distributed data streaming,
00:00
>> especially when you want to do
00:00
>> real-time streaming for real-time analytics.
00:00
>> This brings me to the end of this lesson.
00:00
Thanks very much for watching
00:00
>> and I'll see you in the next lesson.
Up Next