How can an agent resolve this problem? with_retry().batch() returns another input's result when only some inputs recover on retry
Finding: In an external report against langchain-core 1.6.9 installed from master (Python 3.11.15, tenacity 9.1.4), RunnableLambda(...).with_retry(stop_after_attempt=2, wait_exponential_jitter=False) can return the wrong item in batch() and abatch() when one input recovers on retry and another keeps failing. The reporter's example batch of a, b, c gave ['A','A','C'] instead of an exception in b's slot; with return_exceptions=False nothing was raised (https://github.com/langchain-ai/langchain/issues/41162).
Reported mechanism (hypothesis, not maintainer-confirmed): after the last attempt, inputs absent from results_map are filled with result.pop(0). That list is the whole last-attempt output, including successes already stored, so a failed input can take a neighbor's success (https://github.com/langchain-ai/langchain/issues/41162). A commenter says they reproduced the same outputs on a local checkout, including silent wrong output without return_exceptions, and agreed with the diagnosis (https://github.com/langchain-ai/langchain/issues/41162#issuecomment-6072729041). This is an external claim; it is not independent KFA reproduction.
Fix status: the reporter proposes tracking each input's latest exception by original index. A contributor later says they implemented this in #41178 using a last_exception_for dict, that the PR was closed by a bot because of assignment rules, and that it needs reopening; they report tests pass (https://github.com/langchain-ai/langchain/issues/41162#issuecomment-6082729435). No merged commit or released fix appears in the supplied sources, so this should be treated as a proposed fix only.
Practical implication: results from with_retry().batch/abatch on affected master builds may be silently wrong when retry outcomes are mixed, so permanently failing slots should not be trusted. Affected released versions are unknown. No safe workaround is source-backed beyond avoiding this mixed-recovery pattern, which the sources do not validate.
Limitations: all evidence is external report/comment content; KFA did not reproduce it; no maintainer confirmation of cause; fix not shown merged or shipped.
Problem details
- Observed symptom
- A permanently failing input silently receives another input's successful output; with return_exceptions=False no exception is raised.
- Context
- RunnableRetry._batch / _abatch; langchain_core 1.6.9 (installed from master) per reporter; released-version range unknown; RunnableLambda(...).with_retry(stop_after_attempt=2).batch/abatch with mixed recovery (one input recovers on retry, another never does); External reports only; no KFA reproduction.; Root cause is reporter/commenter analysis, not maintainer confirmed.; PR #41178 link and contents not supplied; its tests passing is a self-claim.; Affected released versions unknown; only master-installed 1.6.9 reported.; Comment coverage may omit middle comments.
- Environment
- State
- partial
- Text
- langchain_core 1.6.9 (installed from master) per reporter; released-version range unknown; source-reported environment only
- Symptom signature
- Tool product
- langchain-core
- Component
- RunnableRetry._batch / _abatch
- Operation
- RunnableLambda(...).with_retry(stop_after_attempt=2).batch/abatch with mixed recovery (one input recovers on retry, another never does)
- Literal error text
- No exception raised; reported output ['A','A','C'] instead of ['A', ValueError('b always fails'), 'C']
- Version info
- langchain_core 1.6.9 (installed from master) per reporter; released-version range unknown
- Literal source
- contributor_supplied
- Expected behavior
- Not supplied
Known approaches
None recorded yet.
Sources and related records
No source relations recorded.