Three questions before choosing a service
Inside AWS there are three common ways to move a message around, and the choice comes down to three questions. Must the message be handled by one consumer among many identical ones? Then it is a queue: SQS. Must it reach everyone who wants it? Then it is a topic: SNS. Are the recipients many, do they change often, and must they choose based on the content? Then it is a bus with rules: EventBridge, described in the previous article. They combine: the most common pattern is a topic that feeds one queue per consumer.
There is no AWS account here: I build in miniature the behaviours that drive the decision (visibility timeout, repeated receives, DLQ, per-group ordering, deduplication, fan-out with a filter) and run them with a virtual clock, then I report from the official guide the numbers that matter, consulted on 2026-11-12. It is a model of the documented rules, not the service: whenever the real service behaves differently, or I could not try it, I say so.
The queue: what happens to a message after someone receives it
In SQS a message does not vanish when you receive it: it becomes invisible for the visibility timeout (30 seconds by default, at most 12 hours according to the guide), and it becomes available again if you have not deleted it in the meantime. It is a simple contract: the consumer says it has finished by deleting the message; if it does not, SQS assumes the consumer died and delivers the message again. The model below implements this rule, delivery to the DLQ after a maximum number of receives, and the two FIFO features (order per group, deduplication for five minutes). Time is a counter, so the outputs are reproducible.
// Minimal model of a queue with visibility timeout, DLQ and FIFO mode. Time is virtual (seconds).
type Msg = { id: number; body: string; group?: string; received: number; visibleFrom: number };
class Queue {
msgs: Msg[] = [];
private seen = new Map<string, number>(); // dedupId -> time of the send (FIFO only)
private n = 0;
name: string;
private clock: { t: number };
private opts: { visibility?: number; fifo?: boolean; maxReceive?: number; dlq?: Queue };
constructor(name: string, clock: { t: number }, opts: { visibility?: number; fifo?: boolean; maxReceive?: number; dlq?: Queue } = {}) {
this.name = name; this.clock = clock; this.opts = opts;
}
send(body: string, o: { group?: string; dedupId?: string } = {}): boolean {
if (this.opts.fifo && o.dedupId) {
const before = this.seen.get(o.dedupId);
if (before !== undefined && this.clock.t - before < 300) return false; // 5-minute window
this.seen.set(o.dedupId, this.clock.t);
}
this.msgs.push({ id: ++this.n, body, group: o.group, received: 0, visibleFrom: 0 });
return true;
}
receive(max = 10): Msg[] {
const out: Msg[] = [];
for (const m of [...this.msgs]) {
if (out.length >= max) break;
if (m.visibleFrom > this.clock.t) continue; // still invisible
// FIFO: while a message of the group is in flight, the later ones of the same group stay put.
if (this.opts.fifo && this.msgs.some((x) => x.group === m.group && x.id < m.id)) continue;
if (this.opts.maxReceive !== undefined && m.received >= this.opts.maxReceive && this.opts.dlq) {
this.msgs = this.msgs.filter((x) => x !== m);
this.opts.dlq.msgs.push({ ...m, received: 0, visibleFrom: 0 });
continue; // moved to the DLQ, not delivered
}
m.received++;
m.visibleFrom = this.clock.t + (this.opts.visibility ?? 30);
out.push(m);
}
return out;
}
delete(m: Msg): void { this.msgs = this.msgs.filter((x) => x.id !== m.id); }
}
receive is the method that matters. It skips messages that are still invisible (visibleFrom), counts a receive and
pushes the visibility forward by visibility seconds. If the queue is FIFO, a message is not delivered as long as an
older one of the same group exists: this is how per-group order is guaranteed even with several consumers.
If the message has
already reached maxReceive receives, it ends up in the DLQ instead of being delivered. The messages of group A
wait for A1 even if B is free: groups are the way to get order and parallelism together. One difference from the real
service: the exact moment of the move to the DLQ and the ordering guarantees under heavy load are details of the
guide, not of this model.
Repeated delivery and the poison message
The first scenario shows why an SQS consumer must be idempotent: the work takes 40 seconds and the timeout is 30, so the message is delivered a second time while the first consumer is still working. It happens to slow consumers; it is not an SQS error. The second scenario shows the remedy for messages that can never be processed: after three receives the poison message goes to the DLQ and the good message was never blocked.
const clock = { t: 0 };
// 1) Standard: the consumer is slower than the visibility timeout, the message is delivered twice.
const std = new Queue('std', clock, { visibility: 30 });
std.send('order-1');
const a = std.receive()[0]; // t=0: consumer A starts (it will take 40 s)
clock.t = 31;
const b = std.receive()[0]; // t=31: the timeout has expired, B gets the same message
console.log(`1) A received "${a.body}" at t=0, B receives it at t=31 (receives: ${b.received})`);
// 2) Poison message: it always fails, after maxReceive=3 receives it ends up in the DLQ.
clock.t = 0;
const dlq = new Queue('dlq', clock);
const q = new Queue('orders', clock, { visibility: 30, maxReceive: 3, dlq });
q.send('poison'); q.send('good');
for (let round = 1; round <= 5; round++) {
for (const m of q.receive()) {
if (m.body === 'good') { q.delete(m); console.log(`2) t=${clock.t}: "good" processed`); }
else console.log(`2) t=${clock.t}: "${m.body}" fails (receive ${m.received})`);
}
clock.t += 31;
}
console.log(`2) in queue: ${q.msgs.length}, in DLQ: ${dlq.msgs.map((m) => m.body)}`);
In the first line of output B receives at t=31 the same message that A received at t=0, and the receive counter is 2.
In the second scenario, poison fails at t=0, 31 and 62; on the next round it is no longer delivered and sits in the
DLQ, while good was processed straight away.
The message can stay in the DLQ to be analysed or replayed,
which the article on the dead letter queue covers. The
guide calls maxReceiveCount the number of times a consumer can receive the message before it is
moved: with 1, a single failure is enough to send it to the DLQ, and it should be set higher to tolerate transient faults.
FIFO: order per group and deduplication
The Standard queue guarantees, according to the guide, at-least-once delivery, with possible copies and out-of-order
arrivals, and it is the default choice. The FIFO queue adds order within a group (MessageGroupId)
and deduplication: the guide says that a new send with the same identifier within the five-minute interval does not
introduce duplicates into the queue. Careful: this is not "exactly once" at the consumer level, because a slow
consumer can still receive twice because of the timeout.
// 3) FIFO: order per group and deduplication within 5 minutes.
clock.t = 0;
const fifo = new Queue('orders.fifo', clock, { fifo: true });
const results = [
fifo.send('A1', { group: 'A', dedupId: 'a1' }), fifo.send('A2', { group: 'A', dedupId: 'a2' }),
fifo.send('B1', { group: 'B', dedupId: 'b1' }), fifo.send('A1', { group: 'A', dedupId: 'a1' }),
];
console.log('3) sends accepted:', results);
const r1 = fifo.receive();
console.log('3) first receive:', r1.map((m) => m.body));
r1.filter((m) => m.body === 'A1').forEach((m) => fifo.delete(m));
console.log('3) after deleting A1:', fifo.receive().map((m) => m.body));
The four sends return true, true, true, false: the second send of A1 with the same
dedupId is ignored. The first receive delivers A1 and B1 but not A2, which
waits for A1 to be deleted; after the deletion, A2 comes out. The price of a single group is parallelism: if all the
messages use the same group, only one message at a time is being processed. The guide advises using a different group
for each producer if you need strict order across several producers; choose the group based on the entity that must stay
in order, for example one customer's order, not the whole system.
The topic: copying the message into many queues
An SNS topic keeps nothing for slow consumers: it pushes to all subscribers. That is why it is paired with SQS: each consumer has its own queue subscribed to the topic, and the queue acts as a buffer. The guide says that messages reach the applications that want immediate notification and stay in the queue for those that process them later. A subscription can have a filter policy: a JSON object that defines which messages it receives, evaluated on the message attributes or on the body depending on the chosen scope; without a policy, the subscription receives everything.
// 4) SNS + SQS: a topic copies the message into the subscribed queues whose filter matches.
const inventory = new Queue('inventory', clock), notifications = new Queue('notifications', clock), audit = new Queue('audit', clock);
const subscriptions = [
{ queue: inventory, filter: { eventType: ['OrderCreated'] } },
{ queue: notifications, filter: { eventType: ['OrderCreated', 'OrderShipped'] } },
{ queue: audit, filter: null }, // no filter: receives everything
];
const publish = (eventType: string) => {
for (const s of subscriptions) if (!s.filter || s.filter.eventType.includes(eventType)) s.queue.send(eventType);
};
['OrderCreated', 'OrderShipped', 'OrderCancelled'].forEach(publish);
for (const s of subscriptions) console.log(`4) ${s.queue.name.padEnd(13)}`, s.queue.msgs.map((m) => m.body));
// Output (running the whole file):
// 1) A received "order-1" at t=0, B receives it at t=31 (receives: 2)
// 2) t=0: "poison" fails (receive 1)
// 2) t=0: "good" processed
// 2) t=31: "poison" fails (receive 2)
// 2) t=62: "poison" fails (receive 3)
// 2) in queue: 0, in DLQ: poison
// 3) sends accepted: [ true, true, true, false ]
// 3) first receive: [ 'A1', 'B1' ]
// 3) after deleting A1: [ 'A2' ]
// 4) inventory [ 'OrderCreated' ]
// 4) notifications [ 'OrderCreated', 'OrderShipped' ]
// 4) audit [ 'OrderCreated', 'OrderShipped', 'OrderCancelled' ]
The three subscriptions in the model are those of the typical architecture: inventory wants only created orders,
notifications two types, audit everything because it has no filter. The output shows the copies: three messages
published, one copy in one case, two in another, three in the third. If you get a filter wrong, the message goes to no queue
and there is no error to see, so it is worth always having an audit subscription without a filter. In the model the
filter is an equality on a value; the real service has richer operators, described in the guide, but fewer than those of
EventBridge, which filters on any field of the JSON.
What the guide says: the numbers to know
The numbers below come from the official documentation consulted on 2026-11-12; they change over time and by queue type, so recheck them on the quotas page before sizing.
- The default visibility timeout is 30 seconds, with a maximum of 12 hours from the first receive.
- Long polling waits up to 20 seconds: it reduces empty responses.
- You can send, receive or delete up to 10 messages in a single request.
- A message can have up to 10 attributes and the maximum size is 1 MiB (1,048,576 bytes).
- The default retention is 4 days, and the delivery delay goes up to 15 minutes.
- A Standard queue has a limit of about 120,000 in-flight messages.
The number that bites hardest is the timeout, because it is tuned to the consumer's worst processing time, not to the average: too low produces the double deliveries seen above, too high delays recovery when a consumer dies. The limit on in-flight messages matters when slow consumers and many messages add up; the other values are design parameters. I do not report costs per million messages, latency or maximum throughput: they are numbers that change and that here I can neither measure nor cite from a verified page; for costs, see the AWS pricing page.
The comparison, without figures
| Question | SQS | SNS | EventBridge |
|---|---|---|---|
| Who receives | One consumer among those attached to the queue | All subscribers, with an optional filter | The targets of the rules that match |
| Buffer for slow consumers | Yes, it is its function | No, push delivery (used together with SQS) | Retry per target, optional DLQ |
| Order | Per group, in FIFO queues | In FIFO topics | Stated for the Custom Event Bus; for the Classic the page I consulted does not state it |
| Filter | None | On attributes or body | On any field of the JSON |
| Archive and replay | No: a deleted message cannot be recovered | Not verified here | Yes, documented |
Read the table by row rather than by column. The buffer is the row that separates SQS from everything else: no other combination gives you a queue that absorbs a spike. Order is the row that gets copied wrong most often: SNS also has a FIFO version, and for EventBridge too the guide distinguishes the two bus types.
The filter is the signal for choosing between SNS and EventBridge: if one attribute is enough, the topic is simpler; if you must look at the content, the bus. EventBridge's archive with replay is a reason to choose it even when you do not need the filter, because it lets you feed a new consumer; for SNS I have not verified what the current guide offers, and it must be checked before deciding.
Choosing, in practice
Use SQS when the work must be split among identical consumers or must absorb a spike, or when you need order per group. Use SNS with one queue per consumer when the same event must reach several services and each has its own pace. Use EventBridge when the recipients change often or must choose based on the data, or when you need replay.
The same flow can use all of them: a bus routes by content to one queue per consumer, and the queue holds the buffer and the retries. Whatever the choice, the consumer must be idempotent (see the article on idempotency), because repeated delivery is expected by the service and is not an accident.
What you can do now
You can choose between queue, topic and bus starting from three questions, you can predict what happens to a message when the consumer is slow or fails, and you know why a wrong filter makes messages disappear without errors. You know which numbers to check on the guide and which you do not need. The next step is handling failures: the dead letter queue. For operations that several services must complete together there is the saga, and for the interval in which the data diverges, eventual consistency.
Sources
- Amazon SQS Developer Guide, Amazon SQS standard queues - at-least-once delivery, multiple copies and best-effort order, consulted on 2026-11-12
- Amazon SQS, Visibility timeout - 30 seconds by default, 12 hours maximum, consulted on 2026-11-12
- Amazon SQS, Dead-letter queues - definition of
maxReceiveCountand redrive, consulted on 2026-11-12 - Amazon SQS, FIFO delivery logic - order by message group ID; exactly-once processing - five-minute deduplication interval, consulted on 2026-11-12
- Amazon SQS, Quotas related to messages - maximum size, retention, attributes and batch, consulted on 2026-11-12; short and long polling - maximum wait of 20 seconds
- Amazon SNS Developer Guide, Message filtering - filter policy on attributes or body, without a policy you receive everything, consulted on 2026-11-12; SQS as a subscriber - push delivery from topics and persistence in the queue
The model (queue.ts) was run with Node.js v24.11.0 and the clock is virtual: the outputs are real for the model, not measurements of SQS or SNS. No AWS service was called.
Commenti
Caricamento commenti...
Sign in to comment