#6828 introduced batched vector query, which supports passing multiple query vectors in single query request.
The implementation stores the record batches that referred by topk candidates, which could require m * k * d * 4 bytes at worst case, where m is the number of query vectors, k is the number of results for each query vector, d is the dimension of vector, 4 is sizeof(f32).
We don't need to store the raw vectors unless the query selects vector column, in most cases, vector column won't be selected, so this can significantly reduce the memory footprint
#6828 introduced batched vector query, which supports passing multiple query vectors in single query request.
The implementation stores the record batches that referred by topk candidates, which could require
m * k * d * 4bytes at worst case, wheremis the number of query vectors,kis the number of results for each query vector,dis the dimension of vector,4issizeof(f32).We don't need to store the raw vectors unless the query selects vector column, in most cases, vector column won't be selected, so this can significantly reduce the memory footprint