Skip to content

fix(eval): preserve custom eval metadata#5908

Open
he-yufeng wants to merge 2 commits into
google:mainfrom
he-yufeng:fix/eval-model-extra-metadata
Open

fix(eval): preserve custom eval metadata#5908
he-yufeng wants to merge 2 commits into
google:mainfrom
he-yufeng:fix/eval-model-extra-metadata

Conversation

@he-yufeng
Copy link
Copy Markdown
Contributor

Summary

  • allow eval models to carry custom metadata fields instead of rejecting them as extras
  • cover both top-level EvalCase metadata and nested SessionInput metadata in serialization

Fixes #5906

To verify

  • PYTHONUTF8=1 uv run --with pytest pytest tests/unittests/evaluation/test_eval_case.py -q
  • PYTHONUTF8=1 uv run ruff check src/google/adk/evaluation/common.py tests/unittests/evaluation/test_eval_case.py
  • PYTHONUTF8=1 uv run pyright src/google/adk/evaluation/common.py
  • PYTHONUTF8=1 uv run python -m py_compile src/google/adk/evaluation/common.py tests/unittests/evaluation/test_eval_case.py
  • git diff --check

@adk-bot adk-bot added the eval [Component] This issue is related to evaluation label May 30, 2026
@rohityan rohityan self-assigned this Jun 5, 2026
@adk-bot
Copy link
Copy Markdown
Collaborator

adk-bot commented Jun 5, 2026

I have completed the analysis of Pull Request #5908 ("fix(eval): preserve custom eval metadata"), which resolves Issue #5906.

I verified the contributor's Google CLA signature and performed an architectural/style review against the current repository state:

  • Checked EvalBaseModel baseline which was indeed configured with extra="forbid".
  • Reviewed the PR's implementation of changing this configured value to extra="allow".
  • Assessed the newly added test test_eval_models_preserve_extra_metadata, finding that it adheres to ADK's testing standards.

Please review the comprehensive findings and detailed checklist in our generated report:

👉 pr_analysis_report.md

Key Decisions / Actions For You:

  • No blockers identified: The change is perfectly aligned, safe, and backwards-compatible.
  • Next Steps: You may approve and merge this pull request in the repository. Let me know if you would like

@rohityan
Copy link
Copy Markdown
Collaborator

rohityan commented Jun 5, 2026

/adk-pr-analyze

@adk-bot
Copy link
Copy Markdown
Collaborator

adk-bot commented Jun 5, 2026

I have completed the read-only triage and architectural review of Pull Request #5908 ("fix(eval): preserve custom eval metadata"), which resolves Issue #5906.

Please review the comprehensive findings and detailed checklist in our generated report:

👉 pr_analysis_report.md

Key Highlights & Decisions:

  • No architectural blockers found: The transition of EvalBaseModel's model_config from extra="forbid" to extra="allow" matches target Pydantic patterns perfectly and retains backwards compatibility.
  • Recommended Action: Approve (or Approve with Nits to request adding a behavioral docstring to the new test test_eval_models_preserve_extra_metadata in compliance with Rule 2 of the ADK Testing Style Guide).

@rohityan
Copy link
Copy Markdown
Collaborator

rohityan commented Jun 5, 2026

Hi @he-yufeng , Thank you for your contribution! We appreciate you taking the time to submit this pull request.

@rohityan rohityan added request clarification [Status] The maintainer need clarification or more information from the author and removed request clarification [Status] The maintainer need clarification or more information from the author labels Jun 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

eval [Component] This issue is related to evaluation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

EvalBaseModel extra='forbid' rejects custom fields in session_input and eval cases

3 participants