Skip historical sweep when collectors resume after a gap (erikdarlingdata#892)

When the Off preset, an Agent stoppage, or a server reboot pauses
collection for hours, the next run of query_stats / procedure_stats /
query_store would dump everything that accumulated during the gap into
their delta tables in one go. On query_stats specifically (issue erikdarlingdata#885),
that was enough to blow tempdb overnight.

Each of the three procs now reads MAX(config.collection_log.collection_time)
for its own collector_name (where status = SUCCESS) right after computing
the normal cutoff. If the gap to now exceeds 5x the configured frequency
(or 30 minutes, whichever is larger), it clamps the cutoff to SYSDATETIME()
so only forward-going data is collected on the resume run. NULL/0
frequency_minutes safely floors to 30 minutes.

XE-backed collectors (blocked_process_xml, deadlock_xml, system_health,
default_trace, trace_analysis) are bounded by their own @minutes_back /
@hours_back parameters and don't have the catch-up problem, so they're
left alone. Snapshot collectors (wait_stats, file_io_stats, etc) insert
one row per run regardless of gap and were never at risk.

Verified on sql2016/2017/2019/2022/2025: all three procs deploy cleanly,
heuristic fires on a 3-hour synthetic gap, stays quiet on normal runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: erikdarlingdata

install/08_collect_query_stats.sql

@@ -158,6 +158,42 @@ BEGIN
                         @last_collection_time,
                         DATEADD(MINUTE, -ISNULL(@frequency_minutes, 15), SYSDATETIME())
                     );
+
+            /*
+            Resume detection: if this collector hasn't successfully run in a long time
+            (Off preset, Agent stoppage, server reboot, manual disable), skip the
+            historical sweep so we don't dump the entire plan cache into our deltas.
+            Threshold: 5x the configured frequency, floored at 30 minutes.
+            */
+            DECLARE
+                @last_successful_run_time datetime2(7),
+                @resume_threshold_minutes integer;
+
+            SELECT
+                @last_successful_run_time = MAX(cl.collection_time)
+            FROM config.collection_log AS cl
+            WHERE cl.collector_name = N'query_stats_collector'
+            AND   cl.collection_status = N'SUCCESS';
+
+            SET @resume_threshold_minutes =
+                CASE
+                    WHEN ISNULL(@frequency_minutes, 0) <= 0 THEN 30
+                    WHEN @frequency_minutes * 5 > 30 THEN @frequency_minutes * 5
+                    ELSE 30
+                END;
+
+            IF @last_successful_run_time IS NOT NULL
+            AND DATEDIFF(MINUTE, @last_successful_run_time, SYSDATETIME()) > @resume_threshold_minutes
+            BEGIN
+                IF @debug = 1
+                BEGIN
+                    DECLARE @gap_minutes integer = DATEDIFF(MINUTE, @last_successful_run_time, SYSDATETIME());
+                    RAISERROR(N'Resume detected: %d-minute gap exceeds %d-minute threshold. Skipping historical sweep.', 0, 1,
+                              @gap_minutes, @resume_threshold_minutes) WITH NOWAIT;
+                END;
+
+                SET @cutoff_time = SYSDATETIME();
+            END;
         END;
 
         IF @debug = 1

install/09_collect_query_store.sql

@@ -192,6 +192,44 @@ BEGIN
                         ),
                         0
                     );
+
+            /*
+            Resume detection: if this collector hasn't successfully run in a long time
+            (Off preset, Agent stoppage, server reboot, manual disable), skip the
+            historical sweep so we don't dump the entire Query Store window into our deltas.
+            Note: @last_collection_time above tracks the latest captured query execution time,
+            not the collector's run time, so we need a separate lookup against config.collection_log.
+            Threshold: 5x the configured frequency, floored at 30 minutes.
+            */
+            DECLARE
+                @last_successful_run_time datetime2(7),
+                @resume_threshold_minutes integer;
+
+            SELECT
+                @last_successful_run_time = MAX(cl.collection_time)
+            FROM config.collection_log AS cl
+            WHERE cl.collector_name = N'query_store_collector'
+            AND   cl.collection_status = N'SUCCESS';
+
+            SET @resume_threshold_minutes =
+                CASE
+                    WHEN ISNULL(@collection_interval_minutes, 0) <= 0 THEN 30
+                    WHEN @collection_interval_minutes * 5 > 30 THEN @collection_interval_minutes * 5
+                    ELSE 30
+                END;
+
+            IF @last_successful_run_time IS NOT NULL
+            AND DATEDIFF(MINUTE, @last_successful_run_time, SYSDATETIME()) > @resume_threshold_minutes
+            BEGIN
+                IF @debug = 1
+                BEGIN
+                    DECLARE @gap_minutes integer = DATEDIFF(MINUTE, @last_successful_run_time, SYSDATETIME());
+                    RAISERROR(N'Resume detected: %d-minute gap exceeds %d-minute threshold. Skipping historical sweep.', 0, 1,
+                              @gap_minutes, @resume_threshold_minutes) WITH NOWAIT;
+                END;
+
+                SET @cutoff_time = TODATETIMEOFFSET(SYSUTCDATETIME(), 0);
+            END;
         END;
 
         IF @debug = 1

install/10_collect_procedure_stats.sql

@@ -157,6 +157,42 @@ BEGIN
             SELECT
                 @cutoff_time = ISNULL(@last_collection_time,
                     DATEADD(MINUTE, -ISNULL(@frequency_minutes, 15), SYSDATETIME()));
+
+            /*
+            Resume detection: if this collector hasn't successfully run in a long time
+            (Off preset, Agent stoppage, server reboot, manual disable), skip the
+            historical sweep so we don't dump cumulative procedure stats into our deltas.
+            Threshold: 5x the configured frequency, floored at 30 minutes.
+            */
+            DECLARE
+                @last_successful_run_time datetime2(7),
+                @resume_threshold_minutes integer;
+
+            SELECT
+                @last_successful_run_time = MAX(cl.collection_time)
+            FROM config.collection_log AS cl
+            WHERE cl.collector_name = N'procedure_stats_collector'
+            AND   cl.collection_status = N'SUCCESS';
+
+            SET @resume_threshold_minutes =
+                CASE
+                    WHEN ISNULL(@frequency_minutes, 0) <= 0 THEN 30
+                    WHEN @frequency_minutes * 5 > 30 THEN @frequency_minutes * 5
+                    ELSE 30
+                END;
+
+            IF @last_successful_run_time IS NOT NULL
+            AND DATEDIFF(MINUTE, @last_successful_run_time, SYSDATETIME()) > @resume_threshold_minutes
+            BEGIN
+                IF @debug = 1
+                BEGIN
+                    DECLARE @gap_minutes integer = DATEDIFF(MINUTE, @last_successful_run_time, SYSDATETIME());
+                    RAISERROR(N'Resume detected: %d-minute gap exceeds %d-minute threshold. Skipping historical sweep.', 0, 1,
+                              @gap_minutes, @resume_threshold_minutes) WITH NOWAIT;
+                END;
+
+                SET @cutoff_time = SYSDATETIME();
+            END;
         END;
 
         IF @debug = 1