async_save=true on the default FSDP strategy never writes the checkpoint tracker file, so resume_mode:auto silently restarts from scratch

[AmirF194]

Description

save_checkpoint() in rllm/trainer/verl/utils.py only writes the
latest_checkpointed_iteration.txt tracker file when async_save is False. When
actor.checkpoint.async_save=true is set on the default FSDP strategy, the actor
checkpoint itself is written correctly, but the tracker file is never written. verl's
find_latest_ckpt_path() (the function resume_mode: auto calls to find where to
resume from) has no other way to locate a checkpoint, so it returns None and
training silently starts from scratch, discarding every checkpoint saved so far.

I checked megatron_checkpoint_manager.py in the pinned verl==0.8.0 wheel: it
writes this same tracker file itself once an async Megatron save completes, so
Megatron is self-healing here. fsdp_checkpoint_manager.py has no equivalent, and
rLLM's default strategy is fsdp (rllm/trainer/config/_generated_agent_ppo_trainer.yaml),
so this hits the common path, not an edge case.

Steps to Reproduce

I reproduced this directly against save_checkpoint() (not a full training run), which is
enough to show the tracker file is skipped:

1. Call save_checkpoint(config, global_steps=5, actor_rollout_wg=<a stub whose save_checkpoint() writes files under the given path>, train_dataloader=None)
   twice, once with config.actor_rollout_ref.actor.checkpoint.async_save = False
   and once with True, both pointed at a fresh trainer.default_local_dir.
2. Check for latest_checkpointed_iteration.txt under default_local_dir after each call.

With async_save=False, the tracker file is written. With async_save=True, it is
not, even though the actor checkpoint files are written in both cases.

Error Output / Traceback

No exception. resume_mode: auto just logs "Training from scratch" every time,
because find_latest_ckpt_path() returns None when the tracker file is missing.

rLLM Version

commit 9beb6e0f676a46d38858991fd79ac5f8e0b16d4c (current main)

Training Backend

verl

Additional Context

Fault: rllm/trainer/verl/utils.py, the if not async_save: gate around the
tracker-file write in save_checkpoint() (roughly lines 245-259), and the
matching read side in load_checkpoint().

This exact unconditional gate also exists in vanilla verl's ray_trainer.py, so it
may be worth fixing upstream in verl too rather than only in rLLM's wrapper, or it
may already be a known verl limitation. Happy to send a PR (write the tracker file
regardless of async_save, matching what the Megatron manager already does) if
that's useful, once there's agreement on whether the fix belongs here or upstream.

[cyyecao-lappland]

I'd like to take this one.

I checked the pinned verl==0.8.0 implementation more closely. FSDPCheckpointManager.save_checkpoint() is still synchronous/barriered and does not implement async_save, whereas the Megatron checkpoint manager genuinely schedules an async request and installs a finalize callback that writes latest_checkpointed_iteration.txt only after completion.

So I plan to make the tracker ownership strategy-aware rather than writing it unconditionally:

• fsdp / fsdp2: write the tracker after actor_rollout_wg.save_checkpoint() returns;
• async Megatron: keep letting its completion callback write the tracker, avoiding an early pointer to an incomplete checkpoint;
• add CPU regression tests for sync FSDP, FSDP with async_save=true, async Megatron ownership, and the write ordering.

I'll prepare a PR with that scope. Please let me know if you would prefer this to be handled upstream in verl instead.

[AmirF194]

Thanks for digging in, your read on the FSDP vs async Megatron split matches what I found tracing the checkpoint managers. I will not open a competing PR here, this is yours to take.