[refactor] Remove legacy inference codepath

[SumanthRH]

What does this PR do?

Summary

This PR removes the old Ray actor-based inference stack and standardizes SkyRL on the HTTP/vLLM-based inference path. The _SKYRL_USE_NEW_INFERENCE flag is removed entirely, the legacy inference_engines/ module is deleted, and the remaining training, generation, examples, and docs surfaces now target the HTTP/vLLM server path.

The PR also migrates several examples and integrations that still assumed the old path, fixed/static inference ports, or the legacy custom vLLM server module.

Changes made

Core

Legacy inference stack removal

• Deleted the legacy skyrl/backends/skyrl_train/inference_engines/ package, including the old Ray-wrapped inference client, remote inference engine, HTTP endpoint client, vLLM engine wrapper, and old custom vLLM server.
• Deleted the old inference_servers/vllm_worker.py wrapper.
• Removed _SKYRL_USE_NEW_INFERENCE and collapsed all old/new inference branches onto the HTTP/vLLM path.
• Removed old-inference-only tests, scripts, and CI workflow files.
• Moved the shared inference interface and I/O types into inference_servers/base.py.
• Moved reusable inference helper logic into inference_servers/engine_utils.py.

Inference interface and remote client

• Simplified InferenceEngineInterface around the methods actually used by the HTTP/vLLM path.
• Made RemoteInferenceClient implement the interface directly so the interface and concrete client cannot drift.
• Added interface methods/properties used by generators and integrations:
  • model_name
  • get_endpoint_url()
  • get_world_size()
  • render_chat_completion()
  • finish_session()
• Updated generators and integrations to use InferenceEngineInterface and get_endpoint_url() instead of reaching into RemoteInferenceClient.proxy_url when possible.
• Added coverage that served_model_name is passed into vLLM and used correctly by RemoteInferenceClient and server setup.

vLLM server entrypoint

• Extended skyrl.backends.skyrl_train.inference_servers.vllm_server_actor so it can also be launched as a standalone module:
  • python -m skyrl.backends.skyrl_train.inference_servers.vllm_server_actor
• The standalone entrypoint uses vLLM's OpenAI server setup while preserving SkyRL's custom endpoints needed by training and weight sync.
• Added GPU coverage for the standalone entrypoint:
  • health check
  • OpenAI chat completion
  • SkyRL token-in/token-out generation endpoint
• This lets external rollout-server deployments use the same SkyRL-aware vLLM server path as the Ray actor path.

Weight sync cleanup

• Removed legacy receiver-side abstractions and the unused WeightLoader.
• Simplified weight transfer strategy contracts around the new vLLM-native receiver path.
• Weight loading on the inference side is handled through vLLM's native transfer engines + NewInferenceWorkerWrap.SkyRL's WeightTransferReceiver abstractions have been removed for now.
• Added strategy-to-vLLM transfer engine mapping:
  • broadcast/NCCL -> NCCLWeightTransferEngine
  • CUDA IPC -> IPCWeightTransferEngine
• Simplified BroadcastTransferStrategy.create_init_info by requiring the inference world size from client.get_world_size().
• Removed stale legacy chunk-transfer branches and related tests.

Config and routing cleanup

• Removed old fixed HTTP endpoint config knobs:
  • generator.inference_engine.http_endpoint_host
  • generator.inference_engine.http_endpoint_port
  • generator.inference_engine.enable_http_endpoint
• Cleaned up generator.inference_engine.async_engine references across docs and examples. The config loader now tolerates/removes legacy async_engine=true overrides but rejects unsupported async_engine=false.
• Removed old helper code that only existed for the legacy path, including route_prompts_to_engines and hash_with_sha256.
• Updated SkyRL Agent integration code to stop depending on the removed HTTP endpoint config and old dataloader workarounds.

Tinker updates

• Updated SkyRLTrainBackend so colocated inference engines are put to sleep after initialization, matching the colocated behavior with the python entrypoint and freeing memory before training work starts.
• Updated Tinker E2E scripts to pin the tinker-cookbook commit for reproducibility.
• Updated Tinker/SkyRL sample routing and related tests to work through the HTTP/vLLM path.

Examples

MiniSWE Agent example

• Migrated MiniSWEAgentGenerator to the new inference interface.
• The generator now gets the router URL from inference_engine_client.get_endpoint_url().
• OPENAI_BASE_URL is set at runtime inside the Ray task that runs MiniSWE-Agent, because LiteLLM reads that environment variable from the task process.
• Removed the fixed OPENAI_BASE_URL assumption from .env.miniswe; the router port is dynamic in the new inference path.
• Updated MiniSWE launch scripts to remove legacy inference config overrides.

Thunder-Agent example

• Migrated the Thunder-Agent R2EGym 32B recipe away from the legacy vLLM server module.
• The rollout server launcher now defaults to SkyRL's new standalone vLLM server entrypoint:
  • skyrl.backends.skyrl_train.inference_servers.vllm_server_actor
• This is intentionally not the vanilla vllm.entrypoints.openai.api_server, because SkyRL still needs custom endpoints for weight sync and inference-server control.
• Updated the Thunder-Agent scripts and README around the new rollout-server path.
• Added robust rollout-server process-tree cleanup in the launcher so external vLLM EngineCore / worker subprocesses do not leak GPUs after shutdown.
• Updated Thunder-Agent generator/router handling for the new endpoint semantics and program-release flow.

Remote inference server examples

• Migrated the old remote_inference_engine example into remote_inference_server.
• Added a serve/vLLM-native server launch script for remote inference.
• Updated remote inference docs and examples to point at the serve entrypoint and HTTP/vLLM server path.

Harbor, Verifiers, RLM, and OpenRouter examples

• Updated Harbor and Verifiers generators to use get_endpoint_url() instead of removed fixed host/port config.
• Updated RLM/OpenRouter client code for interface parity with InferenceEngineInterface.
• Removed old enable_http_endpoint / async_engine overrides from Harbor, Verifiers, OpenEnv/OpenReward, RLM, and related integration scripts.

General training scripts

• Removed obsolete generator.inference_engine.async_engine=true and old HTTP endpoint overrides from the example training scripts.
• Migrated the algorithm, Megatron, LoRA, GSM8K, text-to-SQL, search, VisGym, fully async, router replay, and integration launch scripts to the new default inference behavior.
• Updated Tinker example server scripts to use the current serve/vLLM path.

Removed obsolete examples

• Removed the old FlashRL example and docs that depended on the old inference stack.
• Removed old helper scripts that compared old-vs-new inference or launched multiple legacy remote servers.

Docs

Architecture and configuration docs

• Updated the inference architecture docs to describe the HTTP/vLLM path as the standard path.
• Removed references to _SKYRL_USE_NEW_INFERENCE, the old Ray actor inference stack, and deleted legacy classes.
• Updated configuration docs to remove the deprecated HTTP endpoint host/port, enable_http_endpoint, and async_engine knobs.
• Updated placement and Tinker architecture docs for the new colocated inference behavior.

Example docs

• Updated example pages for PPO, Geometry3K, Search, VisGym, multi-turn text-to-SQL, remote inference, and fully async training.
• Removed the FlashRL example page and metadata entry.
• Updated Harbor and agent-integration docs to use the new inference endpoint behavior.

Internal contributor docs

• Updated .claude/docs architecture, inference, contributing, and weight-sync notes to match the new inference and weight-sync structure.
• Updated API/sidebar metadata after deleting or moving example pages.

Tests and CI cleanup

• Deleted legacy-only tests under tests/backends/skyrl_train/inference_engines/.
• Removed old GPU CI workflow/scripts for the legacy inference stack.
• Updated shared tests to run directly against the new inference path.
• Added GPU coverage for the standalone SkyRL vLLM server entrypoint.
• Added/updated coverage for:
  • RemoteInferenceClient
  • served_model_name
  • prompt/chat rendering
  • VLM generation
  • LoRA serving/reload
  • weight sync
  • generator endpoint routing
  • Tinker sample routing

Validation

• ruff and black pass.
• CPU test coverage passes for weight sync, transfer strategies, prompt routing, generators, trainer, config migration, and Tinker sample routing.
• GPU CI workflows pass locally:
  • SkyRL-Train-GPU
  • SkyRL-Train-Megatron-GPU
  • Megatron-Model-GPU-CI
  • SkyRL-GPU-E2E-CI
  • H100 GPU CI
  • SkyRL-GPU-E2E-CI-Tinker-Fully-Async sanity run

[gemini-code-assist]

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.