Commit 9db6dc2
Move SQLite page-out writes to shard writers
Summary:
Today, the page-out workers will flush a sqlite shard if it fills. That stops that worker from finishing anything else that may be in it's queue. When we move the serialization to tokio futures, we do NOT want the tokio async workers pausing for long periods of time to flush sqlite, we want them to continue serializing into the next output buffer so that the sqlite threads can flush on their own later.
--
Replace synchronous sqlite flushes from page-out workers with per-shard `page-sql-N` writer threads and bounded active/spare/pending write buffers. Default the sqlite shard count to 10 while preserving `BUCK2_PAGABLE_SQLITE_SHARDS`, and keep aggregate writer/backpressure counters for validation.
Reviewed By: christolliday
Differential Revision: D109892236
fbshipit-source-id: 589d64d1ba0cc00413f2b5c2f1bbfc18615fd16b1 parent f45aa06 commit 9db6dc2
1 file changed
Lines changed: 383 additions & 43 deletions
0 commit comments