Goal
Add a narrow auto-retry mechanism for transient ACP backend/model failures, especially provider overload at request start.
In practice, provider overload errors such as server_is_overloaded often happen before any assistant output or tool activity is produced. Today the user has to manually send (retry), and session continuity is usually still intact.
Desired behavior
When a prompt fails with a clearly retryable transient backend error before any output has been delivered, acpella should automatically retry once.
Example retryable source error shape:
{
error: {
type: 'service_unavailable_error',
code: 'server_is_overloaded',
message: 'Our servers are currently overloaded. Please try again later.',
},
}A reasonable user-facing flow could be:
Backend overloaded; retrying once...
Then either the retried answer is delivered, or the final error is surfaced normally.
Constraints
Keep the first version conservative:
- Retry only for explicit transient errors such as
server_is_overloaded / service unavailable.
- Retry only if no assistant message chunks have been delivered.
- Retry only if no tool calls have been emitted.
- Max 1 retry initially.
- Use a short backoff.
- Preserve full raw error logging.
Open design question
Retry strategy:
- Replay the original prompt.
- Send a synthetic continuation prompt, e.g.
Please answer the previous user request; the prior response failed due to transient backend overload.
Manual (retry) currently works because the failed user prompt often appears to remain in backend session context. But exact replay may be safer if the failure happened before the backend accepted the prompt. Need to decide based on where acpella can observe the failure point.
Related
Goal
Add a narrow auto-retry mechanism for transient ACP backend/model failures, especially provider overload at request start.
In practice, provider overload errors such as
server_is_overloadedoften happen before any assistant output or tool activity is produced. Today the user has to manually send(retry), and session continuity is usually still intact.Desired behavior
When a prompt fails with a clearly retryable transient backend error before any output has been delivered, acpella should automatically retry once.
Example retryable source error shape:
A reasonable user-facing flow could be:
Then either the retried answer is delivered, or the final error is surfaced normally.
Constraints
Keep the first version conservative:
server_is_overloaded/ service unavailable.Open design question
Retry strategy:
Please answer the previous user request; the prior response failed due to transient backend overload.Manual
(retry)currently works because the failed user prompt often appears to remain in backend session context. But exact replay may be safer if the failure happened before the backend accepted the prompt. Need to decide based on where acpella can observe the failure point.Related