[FlyDSL] [Build] Use upstream parallel scheduler for AOT builds

[zhiding512]

Summary

• replace aiter's duplicate fork pool with flydsl.utils.parallel.run_parallel_jobs across unified and standalone AOT entry points
• preserve deterministic compile failures as structured metadata and aggregate failure causes in unified AOT reporting
• document the upstream scheduler controls and keep setup-time AOT compilation before the CK thread pool

Depends on ROCm/FlyDSL#1038. Merge only after the scheduler is available in the resolved FlyDSL package.

Test plan

• local setup.py -> run_aot -> run_parallel_jobs smoke test with FlyDSL 0.3.1
• FlyDSL focused scheduler tests (20 passed)
• Black and Ruff checks on changed Python files

Made with Cursor

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 4905 --add-label <label>

PR title tags:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf] and op tags like [MLA] are left untouched. Add the no-auto-title label to opt this PR out of title tagging.

[Copilot]

Pull request overview

This PR switches FlyDSL AOT compilation entry points (both unified run_aot and standalone modules) to use FlyDSL’s upstream run_parallel_jobs scheduler instead of aiter’s bespoke fork pool, while preserving deterministic compile failures as structured metadata for unified reporting.

Changes:

• Replace aiter’s custom multiprocessing/file-IPC AOT pool with flydsl.utils.parallel.run_parallel_jobs across unified (aiter.aot.flydsl.common.run_aot) and per-module AOT scripts.
• Add structured failure metadata via compile_failure_info() and aggregate failure causes in unified AOT error reporting.
• Update build-time and documentation notes to reflect upstream scheduler controls and the “fork-before-ThreadPoolExecutor” constraint.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
setup.py	Updates FlyDSL AOT precompile banner/comments and keeps AOT before CK’s thread pool due to fork-based scheduler behavior.
aiter/aot/flydsl/README.md	Documents upstream scheduler usage and updates env var names/behavior to FlyDSL’s controls.
aiter/aot/flydsl/common.py	Removes custom fork pool, adds compile_failure_info, and runs unified AOT via run_parallel_jobs with aggregated failure reporting.
aiter/aot/flydsl/moe.py	Uses run_parallel_jobs and attaches structured failure info on compile exceptions.
aiter/aot/flydsl/mxfp4_moe.py	Attaches structured failure info on compile exceptions.
aiter/aot/flydsl/gemm.py	Uses run_parallel_jobs and attaches structured failure info on compile exceptions.
aiter/aot/flydsl/grouped_moe.py	Uses run_parallel_jobs and returns deterministic failures with structured metadata.
aiter/aot/flydsl/chunk_gdn_h.py	Uses run_parallel_jobs and attaches structured failure info on compile exceptions.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.