Ch. 8 Other Services

SQS - queue service that allows aws services to send data to one another in order to complete jobs (i.e. a message) in a certain order, by pulling messages from SQS

• Example: Search for a flight from AUS to STL, request is sent to EC2 Server and then to SQS, another EC2 instance pulls the message and sends a request to airline servers

• Messages that are partially processed are returned back to SQS queue

• Distributed Messaging System allows components (i.e. aws services) of an app to be Decoupling, because messages that fail/ partially process are returned back to queue

• SQS is pull based, max size is 256KB and can be kept in queue for 1min to 14 days

• Example: EC2 Instance polling the SQS queue for messages for next task to work on

Standard Queue vs. FIFO

• Standard tries its best to deliver messages in FIFO but its not guaranteed, so there will be occasional duplications, but it will deliver a message at least once

• while FIFO is a true queue, and is good for financial transactions that can’t be done out of order or have duplicates

Visibility Timeout - The amount of time a message remains invisible to other services while its being processed, if the processing time goes over the visibility timeout then the message will go back in the queue and be visible for another service to pick up the message/ task.

• Default is 30sec and max is 12 hours

• ChangeMessageVisibility - API Call to change the visibility timeout

Short Polling vs. Long Polling

• Short Polling - a response is returned even if there is isn’t a messages in the SQS queue, and you are changed for empty responses

• Long Polling - only pulls from queue and returns a response when a message is in queue or timeout is reach, and is most cost effective

Managing Large SQS Messages in S3 - large messages over 256KB you will need AWS SDK not CLI and S3

SQS Delay messaging good for large distributed apps that need delays in order for apps not to have issues

What is an SQS Delay Queue? - A Delay Queue is a queue that lets you postpone the delivery of new messages for a number of seconds or minutes.

SNS - notification service that pushes messages (i.e. notifications) only in formats SMS, SQS, HTTPS, email

• Example: Push notification on your phone

Pub Sub Model - users subscribe to a topic (i.e. what sends/ pushes messages) and an admin can publish / push messages to a topic

SNS Example -push notifications

SNS vs SQS

• SNS is for users / components that needs messages pushed to them vs. SQS is for users/ components that need to pull messages from a queue

How could you use Amazon SNS to send immediate push notifications to members on your team that require immediate notification of an event, and also persist those messages for other applications to process at a later time? - You can use SNS in conjunction with SQS to fan a single message out to multiple SQS queues.

SES - email only messaging service, send and receive messages and its not subscription based

Kinesis - group of services that allow you to collect, process, and analyze real time streaming data

• Kinesis Data Steams - capture and store steaming video and data

• Kinesis Video Streams - real time analytics on streaming video data

• Kinesis Firehouse - capture, transform, and load data into AWS data store or other data providers like Splunk

• Kinesis Data Analytics - real time streaming data analytics and need to perform standard SQL queries, data is received from Data Streams, or Firehose

In one example Kinesis Data Streams is used to capture data while DynamoDB is used to store data

Kinesis Shards and Consumers

• Reshard - when you increase the number of shards for your stream

• Kinesis Client Library - creates a record processor on each EC2 instance data consumer, KCL will also distribute record processors equally between consumer instances

  • Ex. Producers are producing 6 shards, and you have 2 instance consumers, so KCL will put 3 record processors on each instance consumer

  • Use an auto scaling group based on CPU load to determine increasing consumers, don’t increase consumers based on resharding

Elastic Beanstalk - web application deployment/ update tool, beanstalk will handle provision AWS resources as well as keep resources like OS and servers up to date and perform health checks, allowing developers to quickly deploy their app or updates without worrying about underlining infrastructure

• Hybrid Control - you can let Beanstalk fully manage EC2 instance or you can take administrative access for certain control

Elastic Beanstalk Deployment Types

• All at Once - Deploys new version of your app to your EC2 instances simultaneously resulting in a total application down while the new version is updating, also will be total application down if you need to roll back, so its not ideal for mission critical apps or apps that need to run 24/7 like google, only good for test/ dev environment

• Rolling - deploys new version in batches as well as potential rollback, during rolling deployment latency will increase since there will be less servers available, so its only ideal for non performance sensitive apps like a local library app

• Rolling with Additional Batch - launches additional batch of instances, and then performs rolling deployment, ideal for maintaining full capacity but not ideal for needing to roll back quickly

• Immutable - launch a whole new equivalent number of instances with the new app version deployed on them, performs health checks, and if new instances are good will replace and delete old instances, ideal for mission critical apps and for fast rollback

• Traffic Splitting - Performs an immutable deployment and then routes some of the traffic to the new servers, if nothing messes up (called canary testing) then all the new instances will replace the old ones

Customize Elastic Beanstalk

• Amazon Linux 1 enviroment

  • .ebestenstions folder only availiabile in Lunux 1, and locationed at top level directory of your app soruce conld containging .config files

• Amazon Linux 2

  • Platform hooks

  • Buildfile

  • Procfile

RDS Deployment with Elastic Beanstalk

• Inside Beanstalk - the pro is that is a quick and east deployment, but if you terminate your beanstalk the DB has to be terminated as well, so only ideal for dev and test

• Outside Beanstalk - you can terminate your app on beanstalk without impacting the DB, and it also takes extra steps to connect your RDS DB with beanstalk, its ideal for prod

Windows Web Application Migration Assiteant for Elastic Beanstalk - Powershell tool that allows you to migrate your in house .NET application or website to AWS Beanstalk

PHP 101

• Server Side Programming language, meaning its for the backend server. Completed to JavaScript which is a frontend programming language