Coupling

Types of coupling that can happen:

Amazon Simple Queue Service (SQS)

Fully managed message queuing service that uses pull mechanism. Not useful for message filtering or long messages.

Message Queue → Temporary repository for messages that wait to be processed, generally small.

• Producer → Component that produces the message and adds it to the queue.
• Consumer → Component that polls the queue for messages and processes them.

General Features:

• Time Stored → Messages can be stored between 1 minute to 14 days (4 days is default). Deleted after maximum retention period.
• Size → Up to 256KB (could be extended to 2GB with a Java library + S3).
• Redundancy → Redundant in multiple AZs in a single region.
• Queue Sharing → Can share with other accounts and can restrict IPs and time of day usage. Queues are independent within a region.
• Encryption → Server-side (only message body) using KMS (both types of queues). To use events in encrypted queues, compatibility needs to be enabled. HIPAA and PCI compliant.
• Cost → Charge per request and data transfer OUT (💵).
• Messages → Global unique ID to track receipts.
  • Can contain metadata attributes.
  • One can Purge the Queue to delete all messages.

Queue types (defined at creation, cannot convert them):

• Standard Queue (⚠️ default queue type)
  • Delivery → At-least-once, might deliver more than one copy.
    • Applications need to be idempotent.
    • Under rare circumstances, you might receive a message you already deleted.
  • Ordering → Best-effort, sometimes the order received might not be the one sent.
  • Throughput → Nearly unlimited.
• FIFO
  • Delivery → First-in-first-out.
  • Ordering → Exact order.
  • Processing → Exactly once. Duplicates not introduced.
  • Throughput → High (300 per sec, 3000 with batching).
  • Message Group → Distinct, ordered groups of messages (has group ID). Within same ID, messages are ordered. Outside same ID might be out of order. Each sender should use a unique ID.
    • No multiple consumers for one group. Use more groups to have more consumers.

Special Features:

• Dead-letter queue (DLQ) support → Queue of messages that couldn’t be processed, it receives them after the maximum number of processing attempts is reached. It’s a normal queue.
  • Redrive policy to lifecycle unconsumed messages from source to DLQ. Then from DLQ to source queue.
• Visibility timeout → Period of time where SQS prevents other consumers from receiving and processing the same message. Default timeout is 30 secs and maximum is 12 hours. Ensures to avoid duplication, a consumer can process and delete the message from the queue.
  • Should be the maximum time the application takes to process and delete a message from the queue.
• Long polling → Queries all the servers for messages, it sends the messages after the maximum number of messages to return is reached or the polling times out. Inexpensive to retrieve as long as the messages are available. Can reduce the cost of SQS as it can reduce the number of empty receives.
  • Short Polling → Enabled by default, polls only a subset of the servers.
  • Good practice to use a single thread to process a single queue.
  • ReceiveMessageWaitTimeSeconds → In Short Polling is set to 0, Long Polling has it set to > 0.

Details:

• It is possible to configure an SNS topic to publish messages to multiple SQS queues (fanout).
• It is possible to configure SQS queues to subscribe to SNS topics.
• The ApproximateAgeOfOldestMessage metric can be used as a reference metric to define scaling policies.

Amazon Simple Notification Service (SNS)