Async file upload retry blocks the event loop

[1fanwang]

Environment details

• Programming language: Python
• OS: macOS
• Language runtime version: 3.12.14
• Package version: 2.22.0 and main at 0ec3d8a4b2c85817434045dad739f6227c2d5c4c

Steps to reproduce

1. Start a local HTTP server that returns 200 without x-goog-upload-status for the first resumable upload request, then returns final on the second request.
2. Call the HTTPX path used by client.aio.files.upload while a second coroutine sleeps for 50 ms.
3. Measure when the second coroutine resumes.

The local server avoids external API and quota dependencies. The upload retry is the real SDK and HTTPX path.

Reproducer

import asyncio
import io
import threading
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

from google.genai import _api_client, types


class Handler(BaseHTTPRequestHandler):
  attempts = 0

  def do_POST(self):
    type(self).attempts += 1
    self.rfile.read(int(self.headers['Content-Length']))
    self.send_response(200)
    if type(self).attempts == 2:
      self.send_header('X-Goog-Upload-Status', 'final')
    self.end_headers()

  def log_message(self, format, *args):
    pass


async def main():
  server = ThreadingHTTPServer(('127.0.0.1', 0), Handler)
  threading.Thread(target=server.serve_forever, daemon=True).start()
  base_url = f'http://127.0.0.1:{server.server_port}'
  client = _api_client.BaseApiClient(
      vertexai=False,
      api_key='test',
      http_options=types.HttpOptions(
          base_url=base_url,
          async_client_args={'trust_env': False},
      ),
  )
  client._use_aiohttp = lambda: False
  _api_client.INITIAL_RETRY_DELAY = 0.25
  started = time.monotonic()

  async def ticker():
    await asyncio.sleep(0.05)
    return time.monotonic() - started

  try:
    _, tick = await asyncio.gather(
        client._async_upload_fd(
            io.BytesIO(b'test'), f'{base_url}/upload', 4
        ),
        ticker(),
    )
    print(f'attempts={Handler.attempts}')
    print(f'ticker_delay={tick:.3f}s')
    print(f'event_loop_blocked={tick >= 0.20}')
  finally:
    await client.aclose()
    server.shutdown()
    server.server_close()


asyncio.run(main())

attempts=2
ticker_delay=0.281s
event_loop_blocked=True

Cancellation is delayed by the same backoff:

attempts=1
cancel_requested_after=0.433s
cancelled_after=0.434s
backoff_blocked_cancellation=True

The HTTPX branch in _async_upload_fd calls time.sleep during retry backoff. This blocks the event loop, so unrelated coroutines and cancellation cannot run until the sleep returns. The aiohttp branch already uses await asyncio.sleep.

Expected behavior: async upload retry backoff should yield to the event loop and remain cancellable.

[Venkaiahbabuneelam]

Hi @1fanwang, Thanks for bringing this to our attention.

I’ve reproduced the issue using the exact code snippet from the users one.
I am escalating this directly to our engineering team to investigate, We appreciate you flagging this!

Thanks