Skip to content

[Frontend][IR][LLVM] Add end-to-end QDQ lowering with roundeven + clamp#575

Open
jb2733 wants to merge 15 commits into
cornell-zhang:mainfrom
jb2733:qdq-roundeven-clamp
Open

[Frontend][IR][LLVM] Add end-to-end QDQ lowering with roundeven + clamp#575
jb2733 wants to merge 15 commits into
cornell-zhang:mainfrom
jb2733:qdq-roundeven-clamp

Conversation

@jb2733

@jb2733 jb2733 commented Mar 19, 2026

Copy link
Copy Markdown

Description

This PR adds end-to-end support for QDQ (Quantize -> Dequantize) lowering in Allo from the PyTorch frontend through IR lowering and LLVM execution.

It lowers torch.quantize_per_tensor(...).dequantize() into explicit fake-quant computation using divide, round-even, zero-point adjustment, clamp, and rescale operations, and verifies numerical agreement with PyTorch.

Problems

Previously, QDQ-style quantization patterns from PyTorch were not fully supported end-to-end. In particular, quantization rounding and clamp behavior were not correctly lowered through the Allo pipeline to LLVM execution.

Proposed Solutions

This PR adds:

  • PyTorch frontend handling for torch.quantize_per_tensor and .dequantize()
  • explicit QDQ lowering into float-domain fake-quant math
  • IR support for roundeven
  • correct lowering of clamp behavior through min / max
  • backend LLVM pipeline fixes and attribute placement needed for successful lowering
  • an end-to-end test validating MLIR generation and numerical agreement with PyTorch

Examples

Input:

  • x = torch.quantize_per_tensor(x, scale, zero_point, dtype).dequantize()

Lowered computation:

  • x = x / scale
  • x = roundeven(x)
  • x = x + zero_point
  • x = max(x, qmin)
  • x = min(x, qmax)
  • x = x - zero_point
  • x = x * scale

Expected behavior:

  • Generated MLIR contains math.roundeven
  • Clamp lowering produces arith.maximumf and arith.minimumf
  • LLVM execution matches PyTorch numerically within tolerance

Checklist

Please make sure to review and check all of these items:

  • PR's title starts with a category (e.g. [Bugfix], [IR], [Builder], etc)
  • All changes have test coverage (It would be good to provide ~2 different test cases to test the robustness of your code)
  • Pass the formatting check locally
  • Code is well-documented

@jb2733 jb2733 closed this Mar 19, 2026
@jb2733 jb2733 reopened this Mar 19, 2026
@jb2733
jb2733 marked this pull request as draft March 19, 2026 16:06
@jb2733
jb2733 marked this pull request as ready for review March 19, 2026 19:21
@jb2733 jb2733 changed the title Qdq roundeven clamp [Frontend][IR][LLVM] Add end-to-end QDQ (quantize_per_tensor → dequantize) lowering with roundeven + clamp Mar 19, 2026
@jb2733 jb2733 changed the title [Frontend][IR][LLVM] Add end-to-end QDQ (quantize_per_tensor → dequantize) lowering with roundeven + clamp [Frontend][IR][LLVM] Add end-to-end QDQ lowering with roundeven + clamp Mar 19, 2026
@chhzh123
chhzh123 requested a review from Copilot March 21, 2026 14:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds end-to-end lowering for PyTorch QDQ (quantize → dequantize) patterns through Allo IR down to LLVM execution, including round-to-even and clamp behavior, with accompanying tests to validate MLIR generation and numeric correctness.

Changes:

  • Add PyTorch frontend lowering for torch.quantize_per_tensor(...).dequantize() into explicit fake-quant math (div → roundeven → zp adjust → clamp → rescale).
  • Extend IR inference/builder to support roundeven/clampf and ensure tensor min/max lower via linalg generic ops.
  • Update backend decomposition/lowering to rewrite math.clampf and adjust LLVM lowering pipeline; add end-to-end tests.

Reviewed changes

Copilot reviewed 7 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tests/test_pytorch_qdq_end_to_end.py New end-to-end tests checking FX graph, MLIR ops (roundeven, maximumf/minimumf), and LLVM vs PyTorch numerics.
tests/test_clampf.py New clampf MLIR + execution test.
allo/passes.py Decompose pass now rewrites math.clampf to arith.maximumf + arith.minimumf prior to backend lowering.
allo/ir/infer.py Type/shape inference support for roundeven and clampf.
allo/ir/builder.py Lower tensor min/max via library path; add library lowering for roundeven/clampf; update scalar external-call operand handling.
allo/frontend/pytorch.py PyTorch FX lowering for quantize_per_tensor into fake-quant math; treat dequantize as identity; add scalar resolution helpers.
allo/backend/llvm.py Adjust LLVM compilation flow/attribute placement and pass pipeline; includes new debug prints.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/test_clampf.py
Comment on lines +6 to +8
torch = pytest.importorskip("torch")
import torch.nn as nn

Copilot AI Mar 21, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test unnecessarily depends on PyTorch: it importorskip("torch") and imports torch.nn, but never uses torch. That will skip the clampf test in environments without torch even though clampf is independent; please remove the torch-related imports/skip here.

Suggested change
torch = pytest.importorskip("torch")
import torch.nn as nn

Copilot uses AI. Check for mistakes.
Comment thread allo/ir/builder.py
"abs": math_d.AbsIOp,
"roundeven": math_d.RoundEvenOp,
"clampf": math_d.ClampFOp,
}.get(fn_name)

Copilot AI Mar 21, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the scalar-math fast path, opcls = {...}.get(fn_name) can be None for an unsupported scalar function name, and the subsequent call will raise a TypeError instead of the intended RuntimeError at the end of this routine. Add an explicit check for opcls is None and raise a clear error before calling it.

Suggested change
}.get(fn_name)
}.get(fn_name)
if opcls is None:
raise RuntimeError(
f"Unsupported scalar math function '{fn_name}' in scalar fast path"
)

Copilot uses AI. Check for mistakes.
Comment thread allo/passes.py
for op in module.body.operations:
if isinstance(op, func_d.FuncOp) and not op.is_external:
rewrite_clampf_in_block(op.entry_block)
if isinstance(op, func_d.FuncOp) and not op.is_external:

Copilot AI Mar 21, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two consecutive checks use the exact same condition (isinstance(op, func_d.FuncOp) and not op.is_external). Consider merging them into a single if block to avoid redundant condition evaluation and make the control flow clearer.

Suggested change
if isinstance(op, func_d.FuncOp) and not op.is_external:

Copilot uses AI. Check for mistakes.
Comment thread allo/backend/llvm.py
Comment on lines +71 to +75
print("=== BEFORE DECOMPOSE ===")
print(self.module)
self.module = decompose_library_function(self.module)
print("=== AFTER DECOMPOSE ===")
print(self.module)

Copilot AI Mar 21, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the unconditional debug prints of the entire MLIR module ("=== BEFORE/AFTER DECOMPOSE ==="). This will spam stdout during normal execution/tests; gate it behind an explicit verbose/debug flag or use the existing logging infrastructure.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants