⚡️ Speed up function replace_none_and_null_with_empty_str by 33% in PR #10567 (feat-agentic-ux)

[codeflash-ai]

⚡️ This pull request contains optimizations for PR #10567

If you approve this dependent PR, these changes will be merged into the original PR branch feat-agentic-ux.

This PR will be automatically closed if the original PR is merged.

---

📄 33% (0.33x) speedup for replace_none_and_null_with_empty_str in src/backend/base/langflow/agentic/mcp/support.py

⏱️ Runtime : 2.69 milliseconds → 2.02 milliseconds (best of 149 runs)

📝 Explanation and details

The optimized code achieves a 33% speedup through several key micro-optimizations that reduce overhead in the critical dictionary processing loop:

Primary Optimizations:

1. Eliminated repeated set creation: The original code created set(required_fields) if required_fields else set() on every function call (line with 974.4ns per hit). The optimized version creates it once as None when not needed, avoiding unnecessary empty set allocation.

2. Precomputed string lookup set: Moved str_nulls = {"null", "nan", "infinity", "-infinity"} outside the inner function to avoid repeated set creation during string comparisons.

3. Optimized type checking flow: Restructured convert_value() with elif chains instead of independent if statements, enabling early returns and reducing redundant type checks on the same value.

4. Removed exception handling overhead: Eliminated the try/except block around math.isnan() since it's now only called on confirmed float types, removing exception handling cost.

5. Conditional required fields processing: Added if required_fields_set: check to skip set operations entirely when no required fields are specified, avoiding the costly required_fields_set - new_dict.keys() operation (which took 672.3ns per hit in the original).

6. Local variable optimization: Used result_append = result = [] to cache the append method for faster lookups in the tight loop.

Performance Impact:
The line profiler shows the main bottleneck is the dictionary comprehension (60.8% → 63.8% of total time), but the optimizations reduced overall function time from 25.5ms to 23.1ms. The improvements are most pronounced in the set operations and conditional logic, which collectively saved ~2.4ms.

Workload Benefits:
These optimizations are particularly effective for:

• Large datasets with many dictionaries (as shown in the 1000-item test cases)
• Cases where required_fields is None/empty (avoids unnecessary set operations)
• Mixed data types requiring frequent type checking in convert_value()

The optimizations maintain identical behavior while reducing computational overhead in data processing pipelines.

✅ Correctness verification report:

Test	Status
⚙️ Existing Unit Tests	🔘 None Found
🌀 Generated Regression Tests	✅ 51 Passed
⏪ Replay Tests	🔘 None Found
🔎 Concolic Coverage Tests	🔘 None Found
📊 Tests Coverage	100.0%

🌀 Generated Regression Tests and Runtime

import math

imports

import pytest
from langflow.agentic.mcp.support import replace_none_and_null_with_empty_str

unit tests

-------------------------------

Basic Test Cases

-------------------------------

def test_none_and_null_string_replacement():
# Test that None and "null" (case-insensitive) are replaced
data = [
{"a": None, "b": "null", "c": "NuLl", "d": 1, "e": "something"},
]
expected = [
{"a": "Not available", "b": "Not available", "c": "Not available", "d": 1, "e": "something"},
]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_nan_float_and_nan_string():
# Test that float('nan') and string "NaN" are replaced
data = [
{"x": float('nan'), "y": "NaN", "z": 0.0},
]
expected = [
{"x": "Not available", "y": "Not available", "z": 0.0},
]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_infinity_and_minus_infinity_strings():
# Test string "Infinity" and "-Infinity" are replaced
data = [
{"a": "Infinity", "b": " -Infinity ", "c": "infinity", "d": "INFINITY", "e": 5},
]
expected = [
{"a": "Not available", "b": "Not available", "c": "Not available", "d": "Not available", "e": 5},
]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_no_replacement_needed():
# Test dicts with no values to replace
data = [
{"a": 1, "b": "hello", "c": 3.14},
]
expected = [
{"a": 1, "b": "hello", "c": 3.14},
]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_required_fields_added():
# Test that missing required fields are added with "Not available"
data = [
{"a": 1},
{"b": 2},
]
required_fields = ["a", "b", "c"]
expected = [
{"a": 1, "b": "Not available", "c": "Not available"},
{"a": "Not available", "b": 2, "c": "Not available"},
]
codeflash_output = replace_none_and_null_with_empty_str(data, required_fields)

-------------------------------

Edge Test Cases

-------------------------------

def test_empty_list():
# Test empty input list
data = []
expected = []
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_empty_dict_in_list():
# Test list with empty dict and required fields
data = [{}]
required_fields = ["x", "y"]
expected = [{"x": "Not available", "y": "Not available"}]
codeflash_output = replace_none_and_null_with_empty_str(data, required_fields)

def test_non_dict_in_list():
# Test that non-dict elements are passed through unchanged
data = [{"a": None}, 123, "hello", None]
expected = [{"a": "Not available"}, 123, "hello", None]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_dict_with_all_fields_none():
# All fields are None
data = [{"a": None, "b": None}]
expected = [{"a": "Not available", "b": "Not available"}]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_whitespace_and_case_variants():
# Test whitespace and case variants of "null", "nan", "infinity"
data = [
{"a": " null ", "b": " NAN ", "c": " Infinity", "d": " -infinity ", "e": " nUlL "},
]
expected = [
{"a": "Not available", "b": "Not available", "c": "Not available", "d": "Not available", "e": "Not available"},
]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_isnat_attribute():
# Test object with isnat attribute set to True
class Dummy:
isnat = True
data = [{"x": Dummy()}]
expected = [{"x": "Not available"}]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_isnat_attribute_false():
# Test object with isnat attribute set to False
class Dummy:
isnat = False
data = [{"x": Dummy()}]
expected = [{"x": data[0]["x"]}]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_float_inf_and_minus_inf_not_replaced():
# Test float('inf') and float('-inf') are NOT replaced
data = [{"a": float('inf'), "b": float('-inf')}]
expected = [{"a": float('inf'), "b": float('-inf')}]
codeflash_output = replace_none_and_null_with_empty_str(data); result = codeflash_output

def test_required_fields_empty():
# Test with required_fields as empty list (should not add anything)
data = [{"a": 1}]
expected = [{"a": 1}]
codeflash_output = replace_none_and_null_with_empty_str(data, [])

def test_required_fields_none():
# Test with required_fields as None (should not add anything)
data = [{"a": 1}]
expected = [{"a": 1}]
codeflash_output = replace_none_and_null_with_empty_str(data, None)

def test_dict_with_extra_fields():
# Test dict with fields not in required_fields
data = [{"a": 1, "b": 2, "c": 3}]
required_fields = ["a"]
expected = [{"a": 1, "b": 2, "c": 3}]
codeflash_output = replace_none_and_null_with_empty_str(data, required_fields)

-------------------------------

Large Scale Test Cases

-------------------------------

def test_large_list_of_dicts():
# Test with 1000 dicts, each with a mix of values
data = [
{"a": None if i % 2 == 0 else i, "b": "null" if i % 3 == 0 else str(i), "c": float('nan') if i % 5 == 0 else i}
for i in range(1000)
]
codeflash_output = replace_none_and_null_with_empty_str(data); result = codeflash_output
# Check that all replacements were made correctly
for i, d in enumerate(result):
if i % 2 == 0:
pass
else:
pass
if i % 3 == 0:
pass
else:
pass
if i % 5 == 0:
pass
else:
pass

def test_large_required_fields():
# Test with many required fields
required_fields = [f"field_{i}" for i in range(50)]
data = [{"field_0": None, "field_1": "null"}]
codeflash_output = replace_none_and_null_with_empty_str(data, required_fields); result = codeflash_output
for i in range(2, 50):
pass

def test_large_dict_with_various_values():
# Test a single dict with 500 keys, alternating between None, "null", float('nan'), and valid
data = [{
f"key_{i}": (
None if i % 4 == 0 else
"null" if i % 4 == 1 else
float('nan') if i % 4 == 2 else
i
)
for i in range(500)
}]
codeflash_output = replace_none_and_null_with_empty_str(data); result = codeflash_output
for i in range(500):
v = result[0][f"key_{i}"]
if i % 4 in (0, 1, 2):
pass
else:
pass

def test_large_list_with_non_dicts():
# Test a large list mixing dicts and non-dicts
data = [{"a": None} if i % 2 == 0 else i for i in range(1000)]
codeflash_output = replace_none_and_null_with_empty_str(data); result = codeflash_output
for i in range(1000):
if i % 2 == 0:
pass
else:
pass

-------------------------------

Mutation-sensitive edge case

-------------------------------

def test_required_fields_added_even_if_dict_empty():
# Test that required_fields are added even if the dict is empty
data = [{}]
required_fields = ["foo", "bar"]
expected = [{"foo": "Not available", "bar": "Not available"}]
codeflash_output = replace_none_and_null_with_empty_str(data, required_fields)

def test_required_fields_not_overwrite_existing():
# Test that required_fields do not overwrite existing keys
data = [{"foo": "hello"}]
required_fields = ["foo", "bar"]
expected = [{"foo": "hello", "bar": "Not available"}]
codeflash_output = replace_none_and_null_with_empty_str(data, required_fields)

def test_string_values_not_replaced_unless_exact():
# Test that "nullify", "nana", "infinite", "-infinite" are NOT replaced
data = [{"a": "nullify", "b": "nana", "c": "infinite", "d": "-infinite"}]
expected = [{"a": "nullify", "b": "nana", "c": "infinite", "d": "-infinite"}]
codeflash_output = replace_none_and_null_with_empty_str(data)

codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

#------------------------------------------------
import math

imports

import pytest
from langflow.agentic.mcp.support import replace_none_and_null_with_empty_str

unit tests

------------------- BASIC TEST CASES -------------------

def test_empty_list_returns_empty_list():
# Should handle empty input gracefully
codeflash_output = replace_none_and_null_with_empty_str([])

def test_single_dict_with_no_special_values():
# Should leave normal values unchanged
data = [{"a": 1, "b": "foo"}]
expected = [{"a": 1, "b": "foo"}]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_replace_none_value():
# Should replace None with "Not available"
data = [{"a": None, "b": 2}]
expected = [{"a": "Not available", "b": 2}]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_replace_null_string_case_insensitive():
# Should replace "null" (case-insensitive) with "Not available"
data = [{"a": "null"}, {"a": "NULL"}, {"a": " Null "}]
expected = [{"a": "Not available"}, {"a": "Not available"}, {"a": "Not available"}]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_replace_nan_string_and_float():
# Should replace "nan" string and float('nan') with "Not available"
data = [{"a": "NaN"}, {"a": float('nan')}]
expected = [{"a": "Not available"}, {"a": "Not available"}]
codeflash_output = replace_none_and_null_with_empty_str(data); result = codeflash_output

def test_replace_infinity_strings():
# Should replace "Infinity" and "-Infinity" strings with "Not available"
data = [{"a": "Infinity"}, {"a": "-Infinity"}, {"a": " infinity "}]
expected = [{"a": "Not available"}, {"a": "Not available"}, {"a": "Not available"}]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_required_fields_adds_missing():
# Should add missing required fields with "Not available"
data = [{"a": 1}]
expected = [{"a": 1, "b": "Not available"}]
codeflash_output = replace_none_and_null_with_empty_str(data, required_fields=["a", "b"])

def test_required_fields_with_existing_fields():
# Should not overwrite existing fields
data = [{"a": 1, "b": 2}]
expected = [{"a": 1, "b": 2}]
codeflash_output = replace_none_and_null_with_empty_str(data, required_fields=["a", "b"])

def test_multiple_dicts_mixed_values():
# Should process each dict independently
data = [
{"a": None, "b": "bar"},
{"a": "null", "b": "baz"},
{"a": 5, "b": float('nan')}
]
expected = [
{"a": "Not available", "b": "bar"},
{"a": "Not available", "b": "baz"},
{"a": 5, "b": "Not available"}
]
codeflash_output = replace_none_and_null_with_empty_str(data)

------------------- EDGE TEST CASES -------------------

def test_non_dict_elements_are_untouched():
# Should leave non-dict elements unchanged
data = [{"a": None}, "notadict", 123, None]
expected = [{"a": "Not available"}, "notadict", 123, None]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_dict_with_empty_string_and_whitespace():
# Should not alter empty or whitespace-only strings (unless they match special values)
data = [{"a": "", "b": " ", "c": "null "}]
expected = [{"a": "", "b": " ", "c": "Not available"}]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_dict_with_boolean_and_zero_values():
# Should not alter boolean or zero values
data = [{"a": False, "b": 0, "c": True}]
expected = [{"a": False, "b": 0, "c": True}]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_dict_with_float_infinity():
# Should NOT replace float('inf') or float('-inf') (only string versions)
data = [{"a": float('inf'), "b": float('-inf')}]
expected = [{"a": float('inf'), "b": float('-inf')}]
codeflash_output = replace_none_and_null_with_empty_str(data); result = codeflash_output

def test_dict_with_custom_object_isnat():
# Should replace if object has .isnat==True
class Dummy:
isnat = True
data = [{"x": Dummy()}]
expected = [{"x": "Not available"}]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_dict_with_custom_object_isnat_false():
# Should not replace if object has .isnat==False
class Dummy:
isnat = False
data = [{"x": Dummy()}]
expected = [{"x": data[0]["x"]}]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_required_fields_empty_list():
# Should not add any fields if required_fields is empty
data = [{"a": 1}]
expected = [{"a": 1}]
codeflash_output = replace_none_and_null_with_empty_str(data, required_fields=[])

def test_required_fields_none():
# Should not add any fields if required_fields is None
data = [{"a": 1}]
expected = [{"a": 1}]
codeflash_output = replace_none_and_null_with_empty_str(data, required_fields=None)

def test_dict_with_nested_dicts():
# Should not recurse into nested dicts
data = [{"a": {"b": None}}]
expected = [{"a": {"b": None}}]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_dict_with_list_value():
# Should not alter list values
data = [{"a": [None, "null", 1]}]
expected = [{"a": [None, "null", 1]}]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_dict_with_tuple_value():
# Should not alter tuple values
data = [{"a": (None, "null", 1)}]
expected = [{"a": (None, "null", 1)}]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_dict_with_set_value():
# Should not alter set values
data = [{"a": {None, "null", 1}}]
expected = [{"a": {None, "null", 1}}]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_required_fields_with_all_special_values():
# Should add missing fields and replace special values
data = [{"a": None}]
expected = [{"a": "Not available", "b": "Not available", "c": "Not available"}]
codeflash_output = replace_none_and_null_with_empty_str(data, required_fields=["a", "b", "c"])

def test_required_fields_with_overlap():
# Should not overwrite present fields, but add missing
data = [{"a": 1, "b": None}]
expected = [{"a": 1, "b": "Not available", "c": "Not available"}]
codeflash_output = replace_none_and_null_with_empty_str(data, required_fields=["a", "b", "c"])

def test_dict_with_unusual_types():
# Should not alter values of unusual types (e.g., bytes)
data = [{"a": b"bytes", "b": bytearray(b"foo")}]
expected = [{"a": b"bytes", "b": bytearray(b"foo")}]
codeflash_output = replace_none_and_null_with_empty_str(data)

------------------- LARGE SCALE TEST CASES -------------------

def test_large_list_of_dicts_all_none():
# Should handle large lists where all values are None
data = [{"a": None, "b": None} for _ in range(500)]
expected = [{"a": "Not available", "b": "Not available"} for _ in range(500)]
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_large_list_of_dicts_mixed_values():
# Should handle large lists with mixed normal and special values
data = []
expected = []
for i in range(500):
if i % 3 == 0:
d = {"a": None, "b": "null", "c": i}
e = {"a": "Not available", "b": "Not available", "c": i}
elif i % 3 == 1:
d = {"a": "foo", "b": float('nan'), "c": "bar"}
e = {"a": "foo", "b": "Not available", "c": "bar"}
else:
d = {"a": i, "b": "baz", "c": None}
e = {"a": i, "b": "baz", "c": "Not available"}
data.append(d)
expected.append(e)
codeflash_output = replace_none_and_null_with_empty_str(data)

def test_large_list_with_required_fields():
# Should fill in missing required fields for large input
data = [{"a": 1} for _ in range(500)]
expected = [{"a": 1, "b": "Not available", "c": "Not available"} for _ in range(500)]
codeflash_output = replace_none_and_null_with_empty_str(data, required_fields=["a", "b", "c"])

def test_large_list_with_non_dict_elements():
# Should leave non-dict elements unchanged in large input
data = [{"a": None}] * 250 + [123] * 250
expected = [{"a": "Not available"}] * 250 + [123] * 250
codeflash_output = replace_none_and_null_with_empty_str(data)

codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

To edit these changes git checkout codeflash/optimize-pr10567-2025-11-14T19.53.40 and push.

[coderabbitai]

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codecov]

Codecov Report

❌ Patch coverage is 11.98257% with 808 lines in your changes missing coverage. Please review.
✅ Project coverage is 30.47%. Comparing base (d107995) to head (7225114).
⚠️ Report is 24 commits behind head on main.

Files with missing lines	Patch %	Lines
...backend/base/langflow/api/utils/mcp/agentic_mcp.py	12.78%	116 Missing ⚠️
...kend/base/langflow/agentic/utils/flow_component.py	0.00%	111 Missing ⚠️
...nd/base/langflow/agentic/utils/component_search.py	0.00%	101 Missing ⚠️
src/backend/base/langflow/agentic/mcp/server.py	0.00%	88 Missing ⚠️
...frontend/src/components/common/assistant/index.tsx	0.00%	71 Missing ⚠️
src/backend/base/langflow/initial_setup/setup.py	11.39%	70 Missing ⚠️
src/frontend/src/stores/assistantManagerStore.ts	0.00%	57 Missing ⚠️
.../backend/base/langflow/agentic/utils/flow_graph.py	0.00%	53 Missing ⚠️
src/backend/base/langflow/agentic/mcp/support.py	0.00%	31 Missing ⚠️
...end/base/langflow/agentic/utils/template_create.py	0.00%	26 Missing ⚠️
... and 12 more

❌ Your patch status has failed because the patch coverage (11.98%) is below the target coverage (40.00%). You can increase the patch coverage or adjust the target coverage.
❌ Your project status has failed because the head coverage (38.94%) is below the target coverage (60.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #10600      +/-   ##
==========================================
- Coverage   31.62%   30.47%   -1.15%     
==========================================
  Files        1348     1362      +14     
  Lines       61098    62006     +908     
  Branches     9129     9167      +38     
==========================================
- Hits        19322    18898     -424     
- Misses      40863    42194    +1331     
- Partials      913      914       +1     

Flag	Coverage Δ
backend	46.81% <13.29%> (-5.03%)	⬇️
frontend	13.51% <0.59%> (-0.09%)	⬇️
lfx	38.94% <91.66%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...c/backend/base/langflow/initial_setup/constants.py	100.00% <100.00%> (ø)
.../frontend/src/controllers/API/helpers/constants.ts	55.55% <ø> (ø)
src/frontend/src/customization/feature-flags.ts	100.00% <100.00%> (ø)
src/lfx/src/lfx/inputs/input_mixin.py	82.68% <100.00%> (+0.15%)	⬆️
src/lfx/src/lfx/inputs/inputs.py	68.37% <100.00%> (ø)
src/lfx/src/lfx/services/settings/constants.py	100.00% <100.00%> (ø)
src/backend/base/langflow/api/v1/endpoints.py	57.00% <85.71%> (-15.15%)	⬇️
src/lfx/src/lfx/services/settings/base.py	69.83% <87.50%> (+0.19%)	⬆️
...end/src/controllers/API/queries/assistant/index.ts	0.00% <0.00%> (ø)
src/backend/base/langflow/agentic/mcp/__main__.py	0.00% <0.00%> (ø)
... and 18 more

... and 54 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

Frontend Unit Test Coverage Report

Coverage Summary

Lines	Statements	Branches	Functions
	14.58% (3984/27312)	7.51% (1560/20764)	8.88% (535/6019)

Unit Test Results

Tests	Skipped	Failures	Errors	Time
1630	0 💤	0 ❌	0 🔥	18.975s ⏱️

[codeflash-ai]

This PR has been automatically closed because the original PR #9357 by rejected-l was closed.