custom filters with async_true enabled giving 'coroutine' object is not subscriptable error

[Spells24]

Description
When applying a custom (synchronous) filter to a passed in object in a for loop, we are getting a TypeError: 'coroutine' object is not subscriptable Error. It seems to be an extension of this issue: #1141

Reproduction
Very similar to the above issue, you can minimally reproduce like:

{% for item in (a.b|add_two(5))[:1] %}{{ item }}{% endfor %}
with a jinja environment of:

jinja_env = jinja2.Environment(enable_async=True)
jinja_env.filters.update({'add_two': lambda x, y: [i+y for i in x]})

rendering with a=dict(b=[1,2,3]) will result in the 'coroutine' object is not subscriptable

We tried removing the parenthesis around the method call:
{% for item in a.b|add_two(5)[:1] %}{{ item }}{% endfor %}
but that results in a different jinja2.exceptions.TemplateSyntaxError: expected token 'end of statement block', got '[' error

Environment

• Python version: 3.9
• Jinja version: 3.0.3

[wynbennett]

I was about to report this issue as well. This is occurring with filters and custom functions. Below is a simple test we used to reproduce both cases:

def load_template(name: str):
    if name == 'filter':
        return '{%- set sorted = (data  | sort)[:1] -%}{{ sorted }}'
    elif name == 'call':
        return '{%- set sorted = external_call(data)[:1] -%}{{ sorted }}'

env = Environment(loader=FunctionLoader(load_template), enable_async=True)
env.globals.update(external_call=lambda lst: sorted(lst))

# Failure with filter
template = env.get_template('filter')
asyncio.run(template.render_async({'data': ['a', 'c', 'b']}))

# Failure with call
template = env.get_template('call')
asyncio.run(template.render_async({'data': ['a', 'c', 'b']}))

Our workaround was to set a custom code_generator_class for our environment. This custom class is a subclass of CodeGenerator. We override the visitor for visit_Call to add the extra ( before the await in self.write("await auto_await(") so it is self.write("(await auto_await("). Of course, we had to add an extra ) at the end as well so the code reads:self.write("))")

We added a similar workaround for the _filter_test_common context manager to make it work for filters.

Here is our workaround code

class CustomCodeGenerator(CodeGenerator):
    @optimizeconst
    def visit_Call(self, node: nodes.Call, frame: Frame, forward_caller: bool = False) -> None:
        if self.environment.is_async:
            self.write("(await auto_await(")
        if self.environment.sandboxed:
            self.write("environment.call(context, ")
        else:
            self.write("context.call(")
        self.visit(node.node, frame)
        extra_kwargs = {"caller": "caller"} if forward_caller else None
        loop_kwargs = {"_loop_vars": "_loop_vars"} if frame.loop_frame else {}
        block_kwargs = {"_block_vars": "_block_vars"} if frame.block_frame else {}
        if extra_kwargs:
            extra_kwargs.update(loop_kwargs, **block_kwargs)
        elif loop_kwargs or block_kwargs:
            extra_kwargs = dict(loop_kwargs, **block_kwargs)
        self.signature(node, frame, extra_kwargs)
        self.write(")")
        if self.environment.is_async:
            self.write("))")

    @contextmanager
    def _filter_test_common(
        self, node: Union[nodes.Filter, nodes.Test], frame: Frame, is_filter: bool
    ) -> Iterator[None]:
        if self.environment.is_async:
            self.write("(await auto_await(")

        if is_filter:
            self.write(f"{self.filters[node.name]}(")
            func = self.environment.filters.get(node.name)
        else:
            self.write(f"{self.tests[node.name]}(")
            func = self.environment.tests.get(node.name)

        # When inside an If or CondExpr frame, allow the filter to be
        # undefined at compile time and only raise an error if it's
        # actually called at runtime. See pull_dependencies.
        if func is None and not frame.soft_frame:
            type_name = "filter" if is_filter else "test"
            self.fail(f"No {type_name} named {node.name!r}.", node.lineno)

        pass_arg = {
            _PassArg.context: "context",
            _PassArg.eval_context: "context.eval_ctx",
            _PassArg.environment: "environment",
        }.get(
            _PassArg.from_obj(func)  # type: ignore
        )

        if pass_arg is not None:
            self.write(f"{pass_arg}, ")

        # Back to the visitor function to handle visiting the target of
        # the filter or test.
        yield

        self.signature(node, frame)
        self.write(")")

        if self.environment.is_async:
            self.write("))")
            
class CustomEnvironment(Environment):
    code_generator_class = CustomCodeGenerator

[davidism]

Thanks for the good examples and investigation. It looks like other parts that use auto_await add surrounding parentheses, but the parts for filters and calls missed it.