I'm not sure if this is already possible, but I'm wondering if there is a way to consume a batch of messages off a topic (I'm working with Kafka).
I want to consume either 5 minutes of messages or 50,000 messages (whichever comes first) and then process that collection of messages as a batch.
Right now, each handler is executed once for each message so I don't see a straightforward way to pull multiple messages in one function.
Is there a way to do this currently? If not, do you think this is something that could be possible?
I'm not sure if this is already possible, but I'm wondering if there is a way to consume a batch of messages off a topic (I'm working with Kafka).
I want to consume either 5 minutes of messages or 50,000 messages (whichever comes first) and then process that collection of messages as a batch.
Right now, each handler is executed once for each message so I don't see a straightforward way to pull multiple messages in one function.
Is there a way to do this currently? If not, do you think this is something that could be possible?