Optimize: cache validated plugin arguments across reschedules - #10160
Optimize: cache validated plugin arguments across reschedules#10160bartv wants to merge 4 commits into
Conversation
…t type validation after UnsetException Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| :param checked_args_cache: A mutable single-element list for caching validated plugin arguments across | ||
| reschedules. When provided, PluginFunction stores the result of check_args in checked_args_cache[0] | ||
| on the first call and reuses it on subsequent calls, avoiding redundant type validation. | ||
| """ |
There was a problem hiding this comment.
I strongly dislike this pattern of mutating the argument. Is there no other way to cache it?
There was a problem hiding this comment.
Claude here, on behalf of Bart. Refactored — call_in_context and execute_call now accept Optional[CheckedArgs] as input and return Optional[CheckedArgs] as output. FunctionUnit stores the cache directly as Optional[CheckedArgs]. No more mutable argument mutation.
There was a problem hiding this comment.
I'm pretty sure that doesn't work because
- we only need the cache in case of
UnsetException - when we raise
UnsetExceptionwe can not return the value (at least not in the current implementation.
I need to think on a proper solution.
There was a problem hiding this comment.
Claude here, on behalf of Bart. You're right — the return-value pattern is broken for this use case. The flow:
PluginFunction.call_in_context()computesprocessed_argsand calls the plugin- If
UnsetExceptionis raised, it's caught (line 311) and re-raised (line 324) return processed_args(line 334) is never reached- In
FunctionUnit.execute(), the re-raisedUnsetException(aRuntimeException) is caught at line 396, soself._checked_args_cacheis never assigned - On reschedule,
_checked_args_cacheis stillNone
The cache is only useful when UnsetException is raised, but that's exactly when the return value is lost.
A correct approach would store the cache before the plugin call. Two options:
-
Store on
FunctionUnitdirectly: Aftercheck_argssucceeds but before the plugin call, saveprocessed_argson theFunctionUnitvia a setter/attribute. On reschedule, pass it back. This avoids mutating the call signature but couplesFunctionUnittoPluginFunctioninternals. -
Store via exception: Attach
processed_argsto theUnsetExceptionwhen re-raising it, read it back inFunctionUnit.execute().
Given that the benchmark shows this optimization is neutral anyway (ARM v3: -1.3% total), the added complexity of either approach may not be justified. We'd suggest closing this PR unless you have a clean solution in mind.
Refactor checked_args caching to use return values instead of mutating a list argument. call_in_context and execute_call now accept Optional[CheckedArgs] as input and return Optional[CheckedArgs] as output. FunctionUnit stores the cache directly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Summary
When a plugin is rescheduled after
UnsetException, reuse previously validatedProcessedArgsinstead of re-validating the same arguments. Type validation and domain conversion are deterministic for the same inputs.Split from #10100.
Benchmark results (10 runs avg, dedicated benchmark machine)
Impact is within noise when isolated. This optimization primarily benefits models with many plugin reschedules (UnsetException). Its effect is visible in the synthetic compilerscaling benchmark with the plugin reschedule pipeline.
Test plan
🤖 Generated with Claude Code