Decoupling Applications with SQS and SNS

Video Activity
Join over 3 million cybersecurity professionals advancing their career
Sign up with
Required fields are marked with an *
or

Already have an account? Sign In »

Time
19 hours 19 minutes
Difficulty
Intermediate
CEU/CPE
20
Video Transcription
00:00
>> Hey, everyone and welcome.
00:00
In this lecture we're going to talk about
00:00
decoupling applications with SQS and SNS.
00:00
Learning objectives are going to be to define
00:00
these two services so
00:00
that you understand a little bit about what they are.
00:00
SQS, otherwise known as Simple Queuing Service,
00:00
is the first and
00:00
the oldest AWS service offer from AWS platform.
00:00
This is the service that kick-started the idea of AWS.
00:00
Amazon needed a queuing service to
00:00
help with the fulfilling of the orders for Amazon.com,
00:00
and so they developed SQS.
00:00
Since then, they were able
00:00
to really refine it and make it really good.
00:00
They decided, you know what?
00:00
Let's see if we can sell this thing
00:00
because there's nothing quite on
00:00
the market at that time and whatever was there
00:00
was a bit too
00:00
expensive and too far out of
00:00
reach for the everyday person.
00:00
They thought, well, we can offer
00:00
this idea and we can
00:00
run it on this thing called the Cloud,
00:00
which is a newer concept.
00:00
They went ahead and did that and pioneered AWS.
00:00
This was the first service and
00:00
soon after that we had SW,
00:00
well, simple workflow service.
00:00
I think it's SWF is the acronym there.
00:00
Then you had S3,
00:00
which we've talked about that so
00:00
that's the storage service.
00:00
They spun up more and more services ever since then.
00:00
SQS, Simple Queuing Service
00:00
is exactly what it sounds like.
00:00
It's a queuing service that allows for
00:00
the queuing of different tasks, different things.
00:00
Maybe you have like an Amazon sense.
00:00
They have a lot of orders that come through
00:00
and they have to fulfill them in
00:00
accordance to a couple of different things,
00:00
the order in which they come in.
00:00
Maybe first-in, first-out, but then
00:00
also making sure that they have
00:00
the inventory in order to fulfill
00:00
said Q or said requests in the Q.
00:00
There's this whole logistical thing going on
00:00
as far as how that whole process works
00:00
outs and how that as well
00:00
grease an oil for Amazon and their business.
00:00
But essentially this is what this is.
00:00
It allows for requests to come in and to
00:00
be ordered and fulfilled in the way that you want it to.
00:00
SQS is a fully managed service
00:00
and it's something that allows for
00:00
decoupling applications because it allows for
00:00
these requests to go through and a medium.
00:00
A medium that's not
00:00
one application directly
00:00
connected to another application,
00:00
or one application that takes
00:00
some requests and it's directly communicating
00:00
with the service or a server.
00:00
If you were to do that,
00:00
the problem with that happening
00:00
is that you could overload
00:00
the server because all
00:00
of these requests are coming through.
00:00
Let's say you have millions of requests coming
00:00
through and the server wouldn't
00:00
be able to handle all the requests.
00:00
What you do is you put a buffer in between
00:00
the application and the server.
00:00
This queuing service allows for
00:00
these requests to be processed and fulfilled in
00:00
a manner that is palatable
00:00
or it is acceptable by the server.
00:00
The server has so many resources,
00:00
of course, you can scale that,
00:00
you can architect or whatever.
00:00
You can use clusters and make this superpower.
00:00
But if you don't have that resource,
00:00
so maybe that's not even enough.
00:00
Maybe you can even fill the amount of requests coming
00:00
through in just that sense
00:00
you can put in a buffer using SQS.
00:00
Here are some attributes.
00:00
You have unlimited throughput,
00:00
meaning that the number of messages
00:00
in the queue, unlimited.
00:00
Default retention for messages is 4-14 days.
00:00
You have low latency,
00:00
which is 10 milliseconds on publish and receive,
00:00
and your limitation of each message
00:00
is 256 kilobytes sent.
00:00
Here is how the messages are sent.
00:00
You can have the messages come in to
00:00
the SQS queue via the software development kits.
00:00
The messages are persisted through
00:00
the SQS until the consumers
00:00
can delete them or deletes them,
00:00
and the retention period it can be deleted or
00:00
the retention period message expires after 14 days,
00:00
and then at that point it goes ahead and deletes itself.
00:00
These are messages, these
00:00
can be types of requests and stuff like that,
00:00
that are eventually going to be processed
00:00
through whatever your workflow is.
00:00
That's totally hypothetical.
00:00
You got to set it up yourself
00:00
and go through that process.
00:00
If you don't understand anything I'm talking about,
00:00
just go review the documentation,
00:00
look up some examples and that should be helpful.
00:00
The best way I can say
00:00
is if you think about the type of logistics
00:00
for a company like Amazon and what they have to go
00:00
through in order to fill this
00:00
all the requests that they get,
00:00
for all of the orders
00:00
that they get across the entire globe,
00:00
they're going to get requests coming
00:00
through and they have to be processed.
00:00
Johnny orders diapers on Amazon
00:00
and it's going to send a message to
00:00
the fulfillment team in the warehouse to go ahead
00:00
and package a box of diapers to send to Johnny's house.
00:00
That process that is partially being done through SQS,
00:00
that messaging to the fulfillment center
00:00
is being done through SQS.
00:00
That's that part of that architecture
00:00
so keep that in mind.
00:00
SQS has a couple of different queuing methods.
00:00
I did mention this earlier,
00:00
so you have FIFO, which is First In, First Out.
00:00
This is going to do exactly what it sounds like.
00:00
It orders the messages and it takes care of
00:00
the fulfillment of set of
00:00
messages in the order in which it was received.
00:00
You have limited throughput though.
00:00
You have 300 messages without batching exactly once
00:00
sends capability and it removes duplicates.
00:00
Keep that in mind because I actually recall seeing
00:00
that as a question when I took my exam.
00:00
Just keep that in mind.
00:00
Messages are processed in the order by consumer.
00:00
SNS is the other service.
00:00
This is a very popular service.
00:00
I've actually used this one.
00:00
SNS is a simple notification service.
00:00
What this does is this sends messages to
00:00
an SNS topic and
00:00
subscribers can subscribe to
00:00
the SNS topic for notifications.
00:00
A good example of this is maybe you have
00:00
SQS fulfilling the queue of requests that come through.
00:00
Johnny wants to make sure that
00:00
he gets an email to make sure
00:00
that the Amazon received his order for diapers.
00:00
This is going to be done within Amazon's application.
00:00
I want to describe to you how
00:00
this works with their stuff.
00:00
Johnny puts it in order for diapers.
00:00
A message gets us to the fulfillment center
00:00
to go ahead and package a box of a diapers
00:00
and begin that mailing
00:00
to Johnny so he gets it in two days.
00:00
Johnny, as soon as he puts in that order,
00:00
he's going to receive an SNS topic notification
00:00
because his email
00:00
automatically subscribes to
00:00
the SNS topic via his account.
00:00
This is something that just naturally happens.
00:00
You receive notifications when your order
00:00
gets put in and when
00:00
your order gets processed and shipped out.
00:00
We all get this.
00:00
I don't know if you can disable
00:00
a notification, you probably can.
00:00
But we all get it it's
00:00
like a native feature that you get.
00:00
When that happens, you are receiving
00:00
a SNS topic notification via email.
00:00
You can also receive SNS topic notifications via texts.
00:00
They do SMS as well.
00:00
Another pretty cool feature.
00:00
You can have up to 10 million subscribers
00:00
per SNS topic, which is quite a lot.
00:00
Then you can have up to a
00:00
100,000 subscriptions per topic.
00:00
Some other things to consider is
00:00
that subscribers can include SQS,
00:00
HTTP requests, and the secure version of that,
00:00
Lambda emails, SMS, and mobile push notifications.
00:00
If you have a mobile app you can use this as well.
00:00
We are about at time.
00:00
To summarize really quick,
00:00
we covered what SQS is and SMS's.
00:00
We talked about SQS, FIFO or FIFO,
00:00
which is First In First Out.
00:00
We talked about the characteristics of
00:00
each service and their differences.
00:00
That wraps up this lecture.
00:00
If you-all have any questions,
00:00
feel free to reach out and if not,
00:00
I will see you in the next lecture.
Up Next