[Frontend][IR][LLVM] Add end-to-end QDQ lowering with roundeven + clamp#575
[Frontend][IR][LLVM] Add end-to-end QDQ lowering with roundeven + clamp#575jb2733 wants to merge 15 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
Adds end-to-end lowering for PyTorch QDQ (quantize → dequantize) patterns through Allo IR down to LLVM execution, including round-to-even and clamp behavior, with accompanying tests to validate MLIR generation and numeric correctness.
Changes:
- Add PyTorch frontend lowering for
torch.quantize_per_tensor(...).dequantize()into explicit fake-quant math (div → roundeven → zp adjust → clamp → rescale). - Extend IR inference/builder to support
roundeven/clampfand ensure tensormin/maxlower via linalg generic ops. - Update backend decomposition/lowering to rewrite
math.clampfand adjust LLVM lowering pipeline; add end-to-end tests.
Reviewed changes
Copilot reviewed 7 out of 9 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_pytorch_qdq_end_to_end.py | New end-to-end tests checking FX graph, MLIR ops (roundeven, maximumf/minimumf), and LLVM vs PyTorch numerics. |
| tests/test_clampf.py | New clampf MLIR + execution test. |
| allo/passes.py | Decompose pass now rewrites math.clampf to arith.maximumf + arith.minimumf prior to backend lowering. |
| allo/ir/infer.py | Type/shape inference support for roundeven and clampf. |
| allo/ir/builder.py | Lower tensor min/max via library path; add library lowering for roundeven/clampf; update scalar external-call operand handling. |
| allo/frontend/pytorch.py | PyTorch FX lowering for quantize_per_tensor into fake-quant math; treat dequantize as identity; add scalar resolution helpers. |
| allo/backend/llvm.py | Adjust LLVM compilation flow/attribute placement and pass pipeline; includes new debug prints. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| torch = pytest.importorskip("torch") | ||
| import torch.nn as nn | ||
|
|
There was a problem hiding this comment.
This test unnecessarily depends on PyTorch: it importorskip("torch") and imports torch.nn, but never uses torch. That will skip the clampf test in environments without torch even though clampf is independent; please remove the torch-related imports/skip here.
| torch = pytest.importorskip("torch") | |
| import torch.nn as nn |
| "abs": math_d.AbsIOp, | ||
| "roundeven": math_d.RoundEvenOp, | ||
| "clampf": math_d.ClampFOp, | ||
| }.get(fn_name) |
There was a problem hiding this comment.
In the scalar-math fast path, opcls = {...}.get(fn_name) can be None for an unsupported scalar function name, and the subsequent call will raise a TypeError instead of the intended RuntimeError at the end of this routine. Add an explicit check for opcls is None and raise a clear error before calling it.
| }.get(fn_name) | |
| }.get(fn_name) | |
| if opcls is None: | |
| raise RuntimeError( | |
| f"Unsupported scalar math function '{fn_name}' in scalar fast path" | |
| ) |
| for op in module.body.operations: | ||
| if isinstance(op, func_d.FuncOp) and not op.is_external: | ||
| rewrite_clampf_in_block(op.entry_block) | ||
| if isinstance(op, func_d.FuncOp) and not op.is_external: |
There was a problem hiding this comment.
These two consecutive checks use the exact same condition (isinstance(op, func_d.FuncOp) and not op.is_external). Consider merging them into a single if block to avoid redundant condition evaluation and make the control flow clearer.
| if isinstance(op, func_d.FuncOp) and not op.is_external: |
| print("=== BEFORE DECOMPOSE ===") | ||
| print(self.module) | ||
| self.module = decompose_library_function(self.module) | ||
| print("=== AFTER DECOMPOSE ===") | ||
| print(self.module) |
There was a problem hiding this comment.
Remove the unconditional debug prints of the entire MLIR module ("=== BEFORE/AFTER DECOMPOSE ==="). This will spam stdout during normal execution/tests; gate it behind an explicit verbose/debug flag or use the existing logging infrastructure.
Description
This PR adds end-to-end support for QDQ (Quantize -> Dequantize) lowering in Allo from the PyTorch frontend through IR lowering and LLVM execution.
It lowers
torch.quantize_per_tensor(...).dequantize()into explicit fake-quant computation using divide, round-even, zero-point adjustment, clamp, and rescale operations, and verifies numerical agreement with PyTorch.Problems
Previously, QDQ-style quantization patterns from PyTorch were not fully supported end-to-end. In particular, quantization rounding and clamp behavior were not correctly lowered through the Allo pipeline to LLVM execution.
Proposed Solutions
This PR adds:
torch.quantize_per_tensorand.dequantize()roundevenmin/maxExamples
Input:
Lowered computation:
Expected behavior:
Checklist
Please make sure to review and check all of these items: