Introduction to Kinesis

Video Activity
Join over 3 million cybersecurity professionals advancing their career
Sign up with
Required fields are marked with an *
or

Already have an account? Sign In »

Time
19 hours 19 minutes
Difficulty
Intermediate
CEU/CPE
20
Video Transcription
00:00
>> Hey everybody and welcome.
00:00
In this lecture, we're going to talk about
00:00
another service called AWS Kinesis.
00:00
Learning objectives are going to be to define Kinesis and
00:00
review the Kinesis firehose and streams,
00:00
which is part of the Kinesis service.
00:00
Basically, AWS Kinesis allows for
00:00
real live stream analysis and processing and
00:00
data collection from videos that would
00:00
typically have to be
00:00
done during a scheduled batch process.
00:00
You can actually do this in real-time and it uses
00:00
machine learning and artificial intelligence
00:00
and all the buzzwords,
00:00
really complicated stuff,
00:00
but it's a fully managed service
00:00
that is offered through AWS Cloud.
00:00
With this, you can collect, you can process,
00:00
you can analyze streaming data,
00:00
really helpful in the media world and stuff like that.
00:00
I think I use case of this,
00:00
when I can't really think of because I'll be honest,
00:00
I've never actually used this one.
00:00
But the one example I can think of is if
00:00
there's a live media going on
00:00
the news and somebody
00:00
says a curse word or something like that.
00:00
You can analyze that in real-time
00:00
and you can bleep it out or something like that.
00:00
That's one instance of when this
00:00
might be useful as probably the worst example.
00:00
[LAUGHTER] There's probably way better uses for this,
00:00
but I'm just thinking of one in
00:00
which this makes sense in my head.
00:00
Hopefully, that helps you.
00:00
If not, I would encourage you to go online and look at
00:00
other methods in which
00:00
you may find this easier to commit to memory.
00:00
But anyways, you can adjust this real-time data
00:00
through application logs, metrics, website, clickstream.
00:00
It's not just video, you can do other things, as well.
00:00
But it does do video as well as the other data streams.
00:00
Here is an example of all the things you can do.
00:00
Are all the different producers here?
00:00
Here on the left-hand side, we have our applications,
00:00
our clients, our SDK, our Kinesis agent.
00:00
Here is all of the producers.
00:00
These are producing the data that we want to have
00:00
analyzed and it's going
00:00
to be sent over to something called a shard.
00:00
A shard is either a or cluster of servers, I believe.
00:00
That is what is going to be processing the data.
00:00
It's going to be collecting it,
00:00
and processing and doing the analysis that's necessary.
00:00
Then it can get a sense over to the consumers for
00:00
additional analysis or if
00:00
it's appending or you're
00:00
going to be working with that data,
00:00
you can do that over there.
00:00
Kinesis has two different options.
00:00
They have the data streams and they have Data firehose.
00:00
Data streams is a streaming service for ingest at scale.
00:00
You can write custom code,
00:00
you can do this in real-time,
00:00
and it's a managed shard splitting service.
00:00
It's all managed.
00:00
You don't have to worry about the infrastructure,
00:00
you can just focus on the actual design
00:00
of that data collection and the architecture within.
00:00
Kinesis data firehose is a little different.
00:00
It's loads the streaming data into
00:00
an S3 bucket and it's also fully managed.
00:00
There's near real-time.
00:00
The buffer is going to be around 60 seconds.
00:00
It's not essentially
00:00
real-time data streams which is real-time.
00:00
There is automatic scaling and there's no data storage.
00:00
Kinesis data analytics is going
00:00
to perform real-time analysis using SQL.
00:00
It's fully managed, it's automatic scaling,
00:00
easy to pay as you go,
00:00
and you can create streams out of the real-time queries.
00:00
To summarize, we briefly covered Kinesis at a high level.
00:00
We discussed the key differences between streams,
00:00
firehose, and data analytics.
00:00
You're not going to likely see a lot of questions on this
00:00
because this is more of a ancillary service,
00:00
something that we want to be aware of,
00:00
but not technically part of the architecture.
00:00
I can say in all of the years I've worked in the Cloud,
00:00
I've never actually worked with this service.
00:00
Never had to secure this service.
00:00
Not that you would have to do a whole lot
00:00
as the Cloud security engineer because this service
00:00
actually is already managed and
00:00
deals with data ingestion and
00:00
processing and all of that good stuff.
00:00
If you're a Cloud security engineer
00:00
and this service happens to be in
00:00
an architecture environment that
00:00
you're tasked to protect, chances are,
00:00
you're going to be more concerned about
00:00
where that data is coming from and how that data
00:00
is being secured on its way in and over
00:00
to whatever storage solution it's going to be going to.
00:00
Those are some thoughts that I have.
00:00
But nevertheless, you want to
00:00
understand what Amazon Kinesis is because
00:00
it's always possible that you may receive a question or
00:00
two on this particular service,
00:00
but I would be very surprised
00:00
if you received more than that.
00:00
That being said, I'm going to go ahead
00:00
and button this lesson up.
00:00
If you have questions,
00:00
feel free to reach out to me.
00:00
Also, be sure to review the documentation
00:00
so that you understand how this is applicable.
00:00
You can speak to it, that way,
00:00
you don't get befuddled when you step in
00:00
for your exam and yes,
00:00
I will go ahead and wrap this up and
00:00
I'll see you in the next lecture.
Up Next