Skip to content

fix: relax memory_kv key validation to allow digits after first letter - #1327

Open
octo-patch wants to merge 1 commit into
ShishirPatil:mainfrom
octo-patch:fix/issue-1272-memory-kv-key-validation
Open

octo-patch wants to merge 1 commit into
ShishirPatil:mainfrom
octo-patch:fix/issue-1272-memory-kv-key-validation

Conversation

@octo-patch

Copy link
Copy Markdown

Fixes #1272

Problem

The _is_valid_key_format method in memory_kv.py used a regex pattern ^[a-z]+(_[a-z0-9]+)*$ that required the first segment of a key to contain only lowercase letters. This incorrectly rejected valid snake_case keys that contain digits after the first character, such as:

  • q1_report (quarterly reports)
  • user123_profile (user profiles with IDs)
  • v2_config (versioned configurations)
  • s3_bucket_name (cloud resource names)

This caused false negative evaluation results during BFCL v4 multi-turn evaluation.

Solution

Updated the regex pattern from:

pattern = r"^[a-z]+(_[a-z0-9]+)*$"

to:

pattern = r"^[a-z][a-z0-9]*(_[a-z0-9]+)*$"

This change:

  • Still requires the first character to be a lowercase letter (no digit-first keys)
  • Allows digits after the first character in any segment
  • Still rejects keys starting with uppercase, digits, spaces, or underscores
  • Aligns with Python PEP 8 naming guidelines and real-world API naming patterns

Testing

Verified with manual tests:

  • q1_report, user123_profile, v2_config, s3_bucket_name → now correctly accepted ✓
  • MyKey, my key, _key, 1key, key_ → still correctly rejected ✓

fixes ShishirPatil#1272)

The previous regex pattern `^[a-z]+(_[a-z0-9]+)*$` required the first
segment to contain only lowercase letters, which incorrectly rejected
valid snake_case keys like 'q1_report', 'user123_profile', and 'v2_config'.

Updated pattern to `^[a-z][a-z0-9]*(_[a-z0-9]+)*$` which:
- Requires the first character to be a lowercase letter
- Allows digits after the first character in any segment
- Still rejects keys starting with digits, uppercase, spaces, or underscores
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BFCL v4] memory_kv key validation rejects valid keys like "q1_report" and "user123_profile"

1 participant