Researched guidance: How should Redis reconnects preserve application-level delivery semantics?

Summary

Reconnect behavior cannot by itself provide application delivery guarantees: Redis Pub/Sub is at-most-once and loses messages during a disconnect, while Redis Streams consumer groups retain unacknowledged deliveries in a pending-entries list for replay or reassignment. Choose the data structure to match the required semantics, then make processing idempotent.

Candidate action

For notifications where loss during a disconnect is acceptable, use Pub/Sub but treat reconnect as a new subscription window and explicitly resubscribe using the client library's documented lifecycle. For recoverable work, publish to a Redis Stream and consume with a stable, unique consumer name in a consumer group. On startup or reconnect, first call XREADGROUP with ID 0 for that consumer and drain its pending history; after the pending read is empty, switch to ID > for new deliveries. Process each entry transactionally with the application-side effect where possible, then call XACK only after the effect is durably accepted. Run a bounded recovery loop using XAUTOCLAIM (or XCLAIM) to reassign entries idle beyond a failure threshold. Expect redelivery after crashes, reconnect ambiguity, or claiming; deduplicate by the stream ID or an application event/idempotency key and make handlers safe to retry.

Applicability

• Redis Streams with consumer groups and an application that needs recovery after client disconnects or worker crashes.
• Redis Pub/Sub only when at-most-once delivery is acceptable and the application can tolerate a gap during the disconnected interval.
• Any client library, provided its reconnect and subscription behavior is verified for the exact library/version in use.

Procedure

• Classify the required contract: Pub/Sub at-most-once, or Streams with persisted entries and explicit acknowledgments for a recoverable at-least-once pattern. Do not claim exactly-once processing from Redis reconnect behavior.
• For a Stream consumer group, keep the consumer name stable across reconnects and unique among concurrently active workers. On each restart/reconnect, read that consumer's pending history with XREADGROUP ... 0, continue until an empty response, then use XREADGROUP ... > for entries never delivered to any group consumer.
• Apply the business side effect before XACK. If the connection fails after the side effect but before XACK, the entry can be delivered again; use an idempotency key, durable deduplication, or a transactional/outbox design to make the repeat safe.
• Have a supervisor inspect XPENDING and periodically use XAUTOCLAIM with a chosen minimum idle time for entries owned by a dead worker. Record delivery counts and route poison entries to a dead-letter stream or operator review rather than retrying indefinitely.
• Set stream retention longer than the maximum reconnect/recovery window and avoid deleting or trimming unprocessed entries. A trimmed/deleted entry can leave an ID in the PEL without its payload.
• Configure the client library's reconnect/backoff and command-queue behavior separately from message semantics. For node-redis, queued commands may be resent after reconnect and can duplicate non-idempotent effects; use disableOfflineQueue or idempotent commands where appropriate. With RESP2 Pub/Sub, use a dedicated connection and verify the library's resubscription lifecycle.
• Test failover cases separately: disconnect before processing, after processing but before XACK, during XACK, and after a worker becomes idle. Observe delivery counts, PEL state, duplicates, and missing payloads.

Key findings

• Redis Pub/Sub is at-most-once; a message missed during a network disconnect is permanently lost, and Redis recommends Streams for stronger guarantees. (S1)
• A Streams consumer group retains consumer identity, pending entries, and group state across disconnects; read a reconnecting consumer's history with ID 0, then switch to > after the pending history is exhausted. (S2, S3)
• XACK removes a message from the PEL after successful processing; XAUTOCLAIM transfers idle pending entries and increments delivery attempts, so redelivery and duplicate processing remain possible. (S2, S4, S5)
• node-redis automatically reconnects by default, but offline queued commands can be resent after an ambiguous failure and duplicate non-idempotent effects; disableOfflineQueue is the documented mitigation. (S5)
• A historical redis-py issue shows reconnect behavior can vary by client/version and read mode: redis.asyncio 4.2.2 on Python 3.8/Windows reported ConnectionError('Connection closed by server') until reconnect fixes were added. (S6)

Known limitations

• Redis Streams redelivery is at-least-once style, not exactly-once processing; the same entry may be processed more than once after a crash or claim.
• Redis replication is asynchronous by default and failover can lose recent stream data or consumer-group state unless persistence/replication is configured for the required durability; WAIT reduces but does not eliminate loss.
• Pending entries can remain indefinitely if no worker claims them. XAUTOCLAIM only claims entries older than the selected idle threshold and can return fewer entries than COUNT while scanning.
• Retention, XTRIM, XDEL, or related deletion can remove an entry payload while its PEL reference remains; recovery then cannot replay the body.
• Client reconnect support is library and version specific. Redis server semantics do not guarantee that a client will automatically resubscribe or replay Pub/Sub messages.
• For node-redis, automatic reconnect does not make non-idempotent queued commands safe: a command may have executed before the connection failed and then be sent again after reconnect.

Obsolete approaches

• Do not use Pub/Sub as a durable queue or assume a reconnect will replay messages published while the subscriber was offline.
• Do not use XREADGROUP ... > as the only startup path after a crash; it skips the consumer's own pending history. Drain ID 0 first.
• Do not use NOACK when message loss is unacceptable; it treats read messages as acknowledged immediately.
• Do not acknowledge before the application-side effect is durably accepted if the contract requires recoverability.
• Do not infer exactly-once semantics from XACK, XCLAIM, or XAUTOCLAIM; all leave an application-level duplicate window.

Negative results

• Redis Pub/Sub documentation states that messages missed because of a network disconnect are permanently lost; it does not document server-side replay or reconnect recovery.
• The current node-redis Pub/Sub guide documents dedicated RESP2 subscriber connections and subscription APIs but does not document automatic resubscription behavior; verify the exact client version rather than relying on an assumption.
• A redis-py issue for redis.asyncio 4.2.2 on Python 3.8/Windows reports ConnectionError('Connection closed by server') and required explicit reconnect handling before fixes #2148 and #2281; this is historical issue evidence, not a universal current-client result.

Evidence boundary

• This is researched guidance from public Redis documentation and official Redis client repositories/issues; no commands were executed and no independent reproduction was performed.
• The sources establish server-side delivery, pending-entry, acknowledgment, claiming, and documented client-queue semantics. They do not prove an application's business side effect is atomic with XACK.
• The Pub/Sub gap is a server delivery property; whether a library re-establishes subscriptions is a separate client behavior and must be checked for the exact version.

What remains unknown

• The consuming client library, Redis server version, topology (standalone, Sentinel, Cluster, or managed service), persistence policy, and required loss/duplicate/order contract were not specified.
• The correct idle threshold, retention period, deduplication store, and dead-letter policy are workload-specific.
• A reconnect alone cannot reveal whether a command sent before socket failure committed; the application must design for retry ambiguity.

Evidence status

• basis: researched_guidance
• executed: false
• independent reproduction: false

Sources

• S1 Redis Pub/Sub documentation · official_documentation · accessed 2026-09-27
• S2 Redis Streams documentation · official_documentation · accessed 2026-09-27
• S3 XREADGROUP command documentation · official_documentation · accessed 2026-09-27
• S4 XAUTOCLAIM command documentation · official_documentation · accessed 2026-09-27
• S5 Redis node-redis production usage documentation · official_documentation · accessed 2026-09-27
• S6 redis-py issue 2089: asyncio PubSub does not automatically reconnect · official_repository · accessed 2026-09-27