Amazon SQS Interview Q & A

Amazon Simple Queue Service (SQS) is a fully managed message queuing service that lets you decouple and scale microservices, distributed systems, and serverless applications by storing messages durably until consumers process them.

A message queue is a temporary storage buffer where producers send messages and consumers read them asynchronously, allowing systems to communicate reliably without being online or available at the same time.

We use SQS to decouple microservices, absorb traffic spikes, retry failed work safely, and ensure that one slow or failed service doesn't break the entire system.

Standard queues offer almost unlimited throughput with at-least-once delivery and best-effort ordering; FIFO queues preserve strict order per message group and support exactly-once processing semantics but with lower, controlled throughput.

A message is a unit of data sent by a producer and read by a consumer; it contains a body (payload) plus optional attributes and SQS metadata like MessageId and ReceiptHandle.

The message body is the actual payload of the message (often JSON or text) that carries the business data your consumer needs to process.

Message attributes are name–type–value metadata fields attached to a message (for example eventType=OrderCreated) that help with filtering, routing, or debugging without parsing the full body.

Visibility timeout is the time period during which a message is hidden from other consumers after one consumer has received it, giving that consumer a chance to process and delete it safely.

The message retention period is how long SQS keeps unconsumed messages in the queue (from 1 minute up to several days); after that time, unprocessed messages are automatically deleted.

Delivery delay is a per-queue or per-message setting that postpones the first time a message becomes visible for consumers, effectively scheduling it for future processing.

By default a single SQS message can be up to 256 KB; for larger payloads you store data in S3 and send only a pointer using the SQS Extended Client Library.

Short polling immediately returns even if no messages are available, while long polling waits up to a configured time (up to 20 seconds) for messages, reducing empty responses, cost, and CPU usage.

It is a CloudWatch/SQS metric that reports the approximate count of messages available for retrieval in the queue, commonly used to monitor backlog and drive autoscaling.

A Dead Letter Queue is a separate queue where messages are moved after they fail processing a configured number of times, allowing you to inspect, fix, and replay problematic messages without blocking normal traffic.

SQS is a pull-based queue used for decoupling and buffering work between producers and consumers, while SNS is a pub/sub notification service that pushes messages to multiple subscribers like SQS, HTTP, Lambda, SMS, or email.

SQS stores messages redundantly across multiple AZs in a region, replicates data automatically, and acknowledges sends only after messages are durably written, which together provide high durability.

The maximum message retention time in SQS is 14 days; after that, any unprocessed messages are removed automatically.

After processing a message, the consumer calls the DeleteMessage (or batch delete) API using the ReceiptHandle received with the message so SQS can remove it from the queue.

If the consumer crashes without deleting the message, the visibility timeout eventually expires and the message becomes visible again so another consumer can retry processing it.

SQS pricing is mainly based on the number of API requests (Send, Receive, Delete, ChangeVisibility), any data transfer between regions, and optional features like SSE encryption charges via KMS.

Visibility timeout is the "processing window" for a message. Example: A Lambda reads an order message and takes ~20 seconds to charge payment and update DB; if visibility timeout is set to 30 seconds, the message stays hidden from other consumers for 30 seconds, giving the Lambda enough time to finish and delete it, otherwise it may be delivered again.

Messages become visible again when their visibility timeout expires without being deleted, or when you explicitly call ChangeMessageVisibility to set the timeout to zero.

Message attributes allow you to attach searchable metadata like eventType, tenantId, or priority, which helps with filtering, routing, analytics, or debugging without parsing the full body.

Retries are handled by letting the message reappear after visibility timeout for another consumer to try, configuring MaxReceiveCount via a redrive policy, and finally moving repeatedly failing messages to a DLQ to avoid infinite loops.

It is a metric showing the age (in seconds) of the oldest message in the queue, which tells you how long messages are waiting before being processed and is a key indicator of backlog or under-provisioned consumers.

You monitor backlog using metrics like ApproximateNumberOfMessagesVisible and ApproximateAgeOfOldestMessage, set CloudWatch alarms on thresholds, and optionally create dashboards to track trends over time.

A redrive policy defines how messages are moved from a source queue to a DLQ; it includes the DLQ ARN and MaxReceiveCount which is the number of failed receives allowed before SQS moves the message.

MaxReceiveCount is the threshold for how many times a message can be received (and not successfully deleted) before SQS sends it to the configured DLQ.

Standard queues provide at-least-once delivery so duplicates are possible; your application must be idempotent. FIFO queues use DeduplicationId and a deduplication window to avoid processing the same logical message multiple times.

SQS is a point-to-point queue mainly for task processing; messages are removed once processed. Kinesis is a streaming service for ordered, replayable data streams used for analytics and real-time processing where multiple consumers can read the same sequence of events.

A message timer allows you to delay a specific message for a set number of seconds (up to 15 minutes), so it only becomes visible after that delay, useful for scheduled or retry-after-X-minutes processing.

If the DLQ is misconfigured, full, or its permissions are wrong, messages may stay in the source queue and keep retrying, so you must monitor DLQ errors and fix configuration; in critical systems, you might add manual alerts or backup logging to avoid silent loss.

You enable server-side encryption (SSE) when creating the queue by specifying a KMS key (AWS-managed or customer-managed); the console, CLI, or CloudFormation all let you set SqsManagedSseEnabled or KmsMasterKeyId and related properties.

A visibility timeout extension is when your consumer calls ChangeMessageVisibility to increase the timeout while still processing a long-running task, preventing the message from being delivered to another consumer too early.

If processing exceeds visibility timeout and you haven't extended it, the message becomes visible again and may be picked up by another Lambda, leading to duplicate processing and possible inconsistent side effects.

You scale consumers by increasing concurrent workers (Lambda concurrency or ECS tasks), using batch receives, and driving autoscaling based on queue depth and age metrics such as ApproximateNumberOfMessagesVisible and ApproximateAgeOfOldestMessage.

A higher batch size improves throughput and reduces cost per message but can increase processing latency and enlarge the impact of a single failure; a lower batch size gives finer-grained control and faster retries but may cost more and limit throughput.

With partial batch response, a Lambda can mark only the failed messages in a batch as not processed; SQS then retries those specific messages while the successfully processed ones are removed, reducing duplicate work.

SQS itself acts as a buffer; producers continue to send messages while consumers process at their own pace, and you control backpressure by adjusting consumer concurrency, batch size, and visibility timeout so downstream systems aren't overloaded.

Standard FIFO queues traditionally support up to a limited number of transactions per second per queue and per MessageGroupId, but with high-throughput FIFO you can achieve much higher TPS while still preserving ordering within each message group.

MessageGroupId defines an ordering group in a FIFO queue; SQS guarantees strict order within a group but can process different groups in parallel, which is how you scale throughput while still preserving ordering where needed.

DeduplicationId is an identifier SQS uses to treat multiple sends as the same logical message within a deduplication window, preventing accidental duplicates caused by retries or network issues.

High Throughput FIFO is an enhanced FIFO queue mode that allows much higher transactions per second by relaxing some throughput limits while still keeping per-MessageGroupId ordering and exactly-once semantics.

FIFO queues combined with idempotent consumers and deduplication (via DeduplicationId) provide exactly-once processing semantics: the queue avoids duplicate delivery inside the window, and the consumer ensures replays don't change state twice.

You store the full payload in S3 and send only a pointer (S3 key, bucket, metadata) in the SQS message, typically using the SQS Extended Client Library to manage this pattern transparently.

Use batch receive, long polling, parallel workers (multi-process, threads, or async), efficient JSON parsing, and bulk writes to downstream systems; also ensure your consumer is idempotent and stateless so you can horizontally scale.

You let a worker poll a batch of messages and then submit each message to a ThreadPoolExecutor or async tasks for IO-bound work like HTTP calls, so multiple messages are processed concurrently within one container or instance.

You choose a FIFO queue and design a consistent MessageGroupId strategy (for example per user, per aggregate, or per order) so all events that must be ordered share the same group, while other groups can run in parallel.

In the transactional outbox pattern, you write domain changes and an outbox table row in the same DB transaction; a background process reliably reads the outbox table and publishes messages to SQS, ensuring you never have "DB committed but no message" inconsistencies.

You can build a DLQ reprocessor (Lambda or batch job) that reads from the DLQ, logs and optionally transforms messages, then sends them back to the main queue or a dedicated "retry" queue after you fix the underlying issue.

You restrict who can send/receive messages using IAM policies and queue policies, and enable SSE with KMS so message payloads are encrypted at rest with fine-grained access control through KMS key policies.

EventBridge rules can target SQS queues; you define a rule that matches specific events (patterns or schedules) and set the SQS queue as a target so matching events are automatically delivered as SQS messages for further processing.

A typical pattern is: data lands in S3 (raw) → events are sent to SQS with file location → Lambda reads messages, processes/cleans data, and writes transformed output to a curated S3 bucket or database, with DLQ handling failures.

You chain multiple queues: one consumer reads from a primary queue, does partial work, and publishes to another queue for the next stage (for example, validate → enrich → persist), allowing each stage to scale independently and be owned by different teams.

Configure a DLQ with a sensible MaxReceiveCount, detect errors that are not retryable, log them clearly, and move or discard such messages rather than letting them re-enter the main queue forever.

SQS automatically scales to accept a surge of messages and acts as a buffer; producers rarely see throttling, while consumers can be scaled gradually based on queue depth and age metrics to catch up safely.

Queue depth tells you how many messages are waiting, while message age tells you how long they have been waiting; together they show whether you're keeping up with throughput or letting messages sit for too long.

Use SNS → SQS fan-out when one event must trigger multiple independent processing flows, each with its own SQS queue (for example, an OrderPlaced event feeding billing, inventory, notifications, and analytics consumers separately).

Use the Extended Client Library when your payloads are frequently larger than 256 KB; it offloads payloads transparently to S3 and stores only references in SQS messages.

You design consumers so that processing the same message twice has no harmful side effects, typically by using idempotency keys, checking for existing operations before applying changes, and keeping a log or table of processed message IDs.