Summary of AWS Application Integration services circa Feb 2019

davidy -

In preparing to take the “AWS Certified Solutions Architect - Professional” exam, I found myself reading through the whitepapers and reference architectures.

The “Overview of Amazon Webservices” whitepaper describes the 140 (!) currently available AWS services, so I thought I’d note down below my own description of each service and how it’s priced.

Table of Contents

This post covers only the “Application Integration” services. See the followup posts in this series for more services ;)

AWS Step Functions (Yahoo Pipes for AWS services)

AWS Step Functions visually join the output of one AWS function (say, a Lambda function) to another (AWS Glue, for example). The Step Function tracks success/failure of each step along the “pipeline”, and can retry/debug failed steps. Step functions comprised of multiple, simple Lamba functions are preferable to single, complex Lamba functions. You pay based on the amount of “state transitions” across your functions, and and the first 4,000 transitions are free.

Amazon MQ (Managed Apache ActiveMQ message broker)

Amazon MQ is a managed instance of Apache ActiveMQ, an open-source message broker supporting advanced features and multiple open standards (MQTT, Stomp, etc). You pay per-instance, and the smallest instance (mq.t2.micro) is free under the Free Tier.

Amazon Simple Queue Service / SQS (Amazon-baked message broker/queuing)

Amazon Simple Queue Service (SQS) is Amazon’s own queuing system (unlike managed Apache MQ, above). It’s serverless, and you pay per-request, starting with 1,000,000 free requests per month. SQS probably offers tighter integration with other AWS services than MQ (for example, AWS Lambda now supports SQS as an event source).

Amazon Simple Notification Service/SNS (Push-based pub/sub many-to-many messaging)

Amazon Simple Notification Service (SNS) is Amazon’s managed pub/sub system, intended to manage the “heavy lifting” of “fanning out” push-based messages to multiple notification recipients. You’d use SNS to power your app’s push notifications, or to send password-reset emails from your website. As with SQS, SNZ is serverless, and you pay per-notification, starting with 1,000,000 free notifications per month, and ~$1 per one million notifications thereafter.

Amazon Simple Workflow Service/SWF (Background task execution/pipelining)

Amazon Simple Workflow Service/SWF manages the background execution of a series of automated tasks. You pay per workflow execution, with 1,000 free workflows to start.

Here’s one of Amazon’s possible use cases:

Video encoding using Amazon S3 and Amazon EC2. In this use case, large videos are uploaded to Amazon S3 in chunks. The upload of chunks has to be monitored. After a chunk is uploaded, it is encoded by downloading it to an Amazon EC2 instance. The encoded chunk is stored to another Amazon S3 location. After all of the chunks have been encoded in this manner, they are combined into a complete encoded file which is stored back in its entirety to Amazon S3. Failures could occur during this process due to one or more chunks encountering encoding errors. Such failures need to be detected and handled through Amazon SWF’s cloud workflow management.