Whats is AWS SQS...?
Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications. SQS eliminates the complexity and overhead associated with managing and operating message oriented middleware, and empowers developers to focus on differentiating work.
With the help of SQS, you can send, store, and receive messages between software components at any volume, without losing messages or requiring other services to be available.
It offers two types of messaging queues :-
1) Standard Queues (default)
2) FIFO Queues (First-In-First-Out)
Standard Queue :-
--> SQS offers a standard queue as the default queue type.
--> It allows you to have an unlimited number of transactions per second.
--> It guarantees that a message is delivered at least once. However, sometime, more than one copy of a message might be delivered out of order.
--> It provides best-effort ordering which ensures that messages are generally delivered in the same order as they are sent but it does not provide a guarantee.
FIFO Queue :-
--> The FIFO Queue complements the standard Queue.
--> It guarantees ordering, i.e., the order in which they are sent is also received in the same order.
--> The most important features of a queue are FIFO Queue and exactly-once processing, i.e., a message is delivered once and remains available until consumer processes and deletes it.
--> It does not allow duplicates to be introduced into the Queue.
-->It also supports message groups that allow multiple ordered message groups within a single Queue.
Benefits of SQS :-
1). Eliminate administrative overhead :-
AWS manages all ongoing operations and underlying infrastructure needed to provide a highly available and scalable message queuing service. With SQS, there is no upfront cost, no need to acquire, install, and configure messaging software, and no time-consuming build-out and maintenance of supporting infrastructure.
2). Reliably deliver messages :-
Use Amazon SQS to transmit any volume of data, at any level of throughput, without losing messages or requiring other services to be available. SQS lets you decouple application components so that they run and fail independently, increasing the overall fault tolerance of the system. Multiple copies of every message are stored redundantly across multiple availability zones so that they are available whenever needed.
3). Keep sensitive data secure :-
You can use Amazon SQS to exchange sensitive data between applications using server-side encryption (SSE) to encrypt each message body. Amazon SQS SSE integration with AWS Key Management Service (KMS) allows you to centrally manage the keys that protect SQS messages along with keys that protect your other AWS resources.
4). Scale elastically and cost-effectively :-
Amazon SQS leverages the AWS cloud to dynamically scale based on demand. SQS scales elastically with your application so you don’t have to worry about capacity planning and pre-provisioning. There is no limit to the number of messages per queue, and standard queues provide nearly unlimited throughput.
Case Study of Oyster Company :-
About Company:-
New York-based Oyster.com shares unvarnished reviews of hotels in nearly 200 destinations worldwide. The company's own investigators visit each location to assess cleanliness, amenities, service and overall quality. What sets Oyster apart from similar sites is its extensive collection of photographs. Oyster takes hundreds of photos for each property, and every review includes dozens of untouched images (submitted by guests as well as investigators) that allow potential visitors to compare a hotel’s marketing material with reality.
Challenge :-
Since its 2009 launch, Oyster has published more than one million high-quality digital images. When this massive volume of images became too cumbersome to handle in-house, the company decided to offload the content to a central repository on Amazon Simple Storage Service (Amazon S3). They migrated to Amazon S3 in 2010 and after that they choose moving to the cloud and Amazon S3 because storing images in their own data center would have been too costly.
Oyster reprocesses its entire collection of photographic images a few times each year to update the copyright year and, if necessary, to change the watermarks. Using their previous solution, reprocessing the entire collection of photographs required about 800 hours to complete.
In addition, they often recreated existing images in new formats and sizes for mobile and tablet devices. Resizing existing images and adding new ones was slowing down the rate at which the company was able to process the collection. It took us more than a week to create new sizes specifically for that resolution. Oyster considered purchasing additional hardware, but found the cost of new hardware and routine maintenance was too high, especially when the machines would sit idle most of the time.
Why Oyster Chooses Amazon Web Services
As they were already using Amazon S3 to store the images, so using Amazon Elastic Compute Cloud (Amazon EC2) to process the images was a natural choice. Chris McBride, a software engineer at Oyster, adds, “We wanted a cloud environment that could be ramped up for the large processing jobs and downsized for the smaller daily jobs.”
While the company is still running one local server, the bulk of the processing work now takes place on the AWS Cloud. Oyster is using a customized Amazon Linux AMI within Amazon EC2. Within this new environment, the company connects to Amazon S3 and Amazon Simple Queue Service (Amazon SQS) using boto, a Python interface to AWS. The images themselves are processed with the ImageMagick software available in the AMI package.
Oyster uses Amazon EC2 instances and Amazon SQS in an integrated workflow to generate the sizes they need for each photo. The team processes a few thousand photos each night, using Amazon EC2 Spot Instances. When Oyster processes the entire collection, it can use up to 100 Amazon EC2 instances. The team uses Amazon SQS to communicate the photos that need to be processed and the status of the jobs.
Hope you will get an idea how oyster uses AWS SQS
Thanks for Reading ✌✌
Comments