Skip to content

Add nonclustered indexes for query/procedure/query store lookups #859

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
erikdarlingdata merged 1 commit into from
Apr 17, 2026
Merged

Add nonclustered indexes for query/procedure/query store lookups #859

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions Dashboard/Services/DatabaseService.QueryPerformance.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1123,7 +1123,7 @@ USE HINT('ENABLE_PARALLEL_PLAN_PREFERENCE')

SELECT TOP (500)
database_name = pl.database_name,
query_hash = CONVERT(nvarchar(20), pl.query_hash, 1),
query_hash = pl.query_hash,
object_type = MAX(pl.object_type),
object_name =
CASE MAX(pl.object_type)
Expand Down Expand Up @@ -1180,7 +1180,7 @@ HASH GROUP
/*Phase 3: hydrate text and plan XML for the TOP 500 winners only*/
SELECT
tr.database_name,
tr.query_hash,
query_hash = CONVERT(nvarchar(20), tr.query_hash, 1),
tr.object_type,
tr.object_name,
tr.first_execution_time,
Expand Down Expand Up @@ -1224,7 +1224,7 @@ OUTER APPLY
SELECT TOP (1)
query_text = CAST(DECOMPRESS(qs2.query_text) AS nvarchar(max))
FROM collect.query_stats AS qs2
WHERE qs2.query_hash = CONVERT(binary(8), tr.query_hash, 1)
WHERE qs2.query_hash = tr.query_hash
Copy link
Copy Markdown
Owner Author

@erikdarlingdata erikdarlingdata Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix looks right: tr.query_hash now stays binary(8) through Phase 2/3 and only converts to nvarchar(20) in the final projection (line 1183), so qs2.query_hash = tr.query_hash is a native binary = binary comparison and sargable against the new IX_query_stats_hash_lookup. The symmetric fix is applied in the MCP copy of the query at lines 4441/4484. No other OUTER APPLYs in this file regressed (the remaining CONVERT(binary(8), @param, 1) forms at 2754/2923 are parameter→constant conversions, still sargable).

Generated by Claude Code

AND qs2.database_name = tr.database_name
ORDER BY qs2.collection_time DESC
) AS qt
Expand All @@ -1233,7 +1233,7 @@ OUTER APPLY
SELECT TOP (1)
query_plan_xml = CAST(DECOMPRESS(qs3.query_plan_text) AS nvarchar(max))
FROM collect.query_stats AS qs3
WHERE qs3.query_hash = CONVERT(binary(8), tr.query_hash, 1)
WHERE qs3.query_hash = tr.query_hash
AND qs3.database_name = tr.database_name
AND qs3.query_plan_text IS NOT NULL
ORDER BY qs3.collection_time DESC
Expand Down Expand Up @@ -4379,7 +4379,7 @@ USE HINT('ENABLE_PARALLEL_PLAN_PREFERENCE')

SELECT TOP (@top + 5)
database_name = pl.database_name,
query_hash = CONVERT(nvarchar(20), pl.query_hash, 1),
query_hash = pl.query_hash,
object_type = MAX(pl.object_type),
object_name =
CASE MAX(pl.object_type)
Expand Down Expand Up @@ -4438,7 +4438,7 @@ HASH GROUP
/*Phase 3: hydrate text for winners only, apply WAITFOR filter*/
SELECT TOP (@top)
tr.database_name,
tr.query_hash,
query_hash = CONVERT(nvarchar(20), tr.query_hash, 1),
tr.object_type,
tr.object_name,
tr.first_execution_time,
Expand Down Expand Up @@ -4481,7 +4481,7 @@ OUTER APPLY
SELECT TOP (1)
query_text = CAST(DECOMPRESS(qs2.query_text) AS nvarchar(max))
FROM collect.query_stats AS qs2
WHERE qs2.query_hash = CONVERT(binary(8), tr.query_hash, 1)
WHERE qs2.query_hash = tr.query_hash
AND qs2.database_name = tr.database_name
ORDER BY qs2.collection_time DESC
) AS qt
Expand Down
78 changes: 78 additions & 0 deletions install/02_create_tables.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1495,5 +1495,83 @@ BEGIN
PRINT 'Created collect.server_properties table';
END;

/*
Nonclustered indexes to support OUTER APPLY lookups in the Dashboard's
query/procedure/query store grid queries. Without these, the optimizer builds
an Eager Index Spool over the entire table to service the lookups, which can
take minutes on large installations (see #835).

ONLINE = ON is only supported on Enterprise/Developer/Azure editions. The
options string is built dynamically based on SERVERPROPERTY('EngineEdition').
*/
DECLARE @online_option nvarchar(20) =
Copy link
Copy Markdown
Owner Author

@erikdarlingdata erikdarlingdata Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: consider a GO before this DECLARE to place it in its own batch. The preceding tables are wrapped in IF NOT EXISTS … BEGIN … END; blocks without intervening GOs, so @online_option/@index_sql land in whatever batch the install runner is composing. Not a correctness issue today (no name collisions), but isolating DDL-generation variables keeps future edits safer.

Generated by Claude Code

CASE
WHEN CAST(SERVERPROPERTY(N'EngineEdition') AS integer) IN (3, 5, 8)
THEN N', ONLINE = ON'
ELSE N''
END;

Copy link
Copy Markdown
Owner Author

@erikdarlingdata erikdarlingdata Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EngineEdition 3 covers Enterprise and Developer (they share the value), 5 = Azure SQL Database, 8 = Managed Instance. That matches where ONLINE = ON is available. DATA_COMPRESSION = PAGE is unconditional here — fine for the supported matrix (SQL 2016 SP1+ made row/page compression a Standard feature), just flagging that this raises the minimum to SP1 for Standard installs.

Generated by Claude Code

DECLARE @index_sql nvarchar(max);

IF NOT EXISTS
(
SELECT
1/0
FROM sys.indexes
WHERE object_id = OBJECT_ID(N'collect.query_stats')
AND name = N'IX_query_stats_hash_lookup'
)
BEGIN
SET @index_sql = N'
CREATE NONCLUSTERED INDEX
IX_query_stats_hash_lookup
ON collect.query_stats
(query_hash, database_name, collection_time DESC)
WITH
(SORT_IN_TEMPDB = ON, DATA_COMPRESSION = PAGE' + @online_option + N');';
EXEC sys.sp_executesql @index_sql;
PRINT 'Created collect.query_stats.IX_query_stats_hash_lookup index';
END;

IF NOT EXISTS
(
SELECT
1/0
FROM sys.indexes
WHERE object_id = OBJECT_ID(N'collect.procedure_stats')
AND name = N'IX_procedure_stats_name_lookup'
)
BEGIN
SET @index_sql = N'
CREATE NONCLUSTERED INDEX
IX_procedure_stats_name_lookup
ON collect.procedure_stats
(database_name, schema_name, object_name, collection_time DESC)
WITH
(SORT_IN_TEMPDB = ON, DATA_COMPRESSION = PAGE' + @online_option + N');';
EXEC sys.sp_executesql @index_sql;
PRINT 'Created collect.procedure_stats.IX_procedure_stats_name_lookup index';
END;

IF NOT EXISTS
(
SELECT
1/0
FROM sys.indexes
WHERE object_id = OBJECT_ID(N'collect.query_store_data')
AND name = N'IX_query_store_data_id_lookup'
)
BEGIN
SET @index_sql = N'
CREATE NONCLUSTERED INDEX
IX_query_store_data_id_lookup
ON collect.query_store_data
(database_name, query_id, collection_time DESC)
WITH
(SORT_IN_TEMPDB = ON, DATA_COMPRESSION = PAGE' + @online_option + N');';
EXEC sys.sp_executesql @index_sql;
PRINT 'Created collect.query_store_data.IX_query_store_data_id_lookup index';
END;

PRINT 'All collection tables created successfully';
GO
98 changes: 98 additions & 0 deletions upgrades/2.7.0-to-2.8.0/01_add_query_lookup_indexes.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
/*
Copyright 2026 Darling Data, LLC
https://www.erikdarling.com/

Upgrade from 2.7.0 to 2.8.0

Add nonclustered indexes on collect.query_stats, collect.procedure_stats, and
collect.query_store_data to support OUTER APPLY lookups in the Dashboard's
grid queries. Without these, the optimizer builds an Eager Index Spool over
the entire table to service the lookups, which can take minutes on large
installations (see #835).

ONLINE = ON is only supported on Enterprise/Developer/Azure editions. The
options string is built dynamically based on SERVERPROPERTY('EngineEdition').
*/

SET ANSI_NULLS ON;
SET ANSI_PADDING ON;
SET ANSI_WARNINGS ON;
SET ARITHABORT ON;
SET CONCAT_NULL_YIELDS_NULL ON;
SET QUOTED_IDENTIFIER ON;
SET NUMERIC_ROUNDABORT OFF;
SET IMPLICIT_TRANSACTIONS OFF;
SET STATISTICS TIME, IO OFF;
GO

USE PerformanceMonitor;
GO

DECLARE @online_option nvarchar(20) =
CASE
WHEN CAST(SERVERPROPERTY(N'EngineEdition') AS integer) IN (3, 5, 8)
THEN N', ONLINE = ON'
ELSE N''
END;

DECLARE @index_sql nvarchar(max);

IF NOT EXISTS
(
SELECT
1/0
FROM sys.indexes
WHERE object_id = OBJECT_ID(N'collect.query_stats')
AND name = N'IX_query_stats_hash_lookup'
)
BEGIN
SET @index_sql = N'
CREATE NONCLUSTERED INDEX
IX_query_stats_hash_lookup
ON collect.query_stats
(query_hash, database_name, collection_time DESC)
WITH
(SORT_IN_TEMPDB = ON, DATA_COMPRESSION = PAGE' + @online_option + N');';
EXEC sys.sp_executesql @index_sql;
PRINT 'Created collect.query_stats.IX_query_stats_hash_lookup index';
END;

IF NOT EXISTS
(
SELECT
1/0
FROM sys.indexes
WHERE object_id = OBJECT_ID(N'collect.procedure_stats')
AND name = N'IX_procedure_stats_name_lookup'
)
BEGIN
SET @index_sql = N'
CREATE NONCLUSTERED INDEX
IX_procedure_stats_name_lookup
ON collect.procedure_stats
(database_name, schema_name, object_name, collection_time DESC)
WITH
(SORT_IN_TEMPDB = ON, DATA_COMPRESSION = PAGE' + @online_option + N');';
EXEC sys.sp_executesql @index_sql;
PRINT 'Created collect.procedure_stats.IX_procedure_stats_name_lookup index';
END;

IF NOT EXISTS
(
SELECT
1/0
FROM sys.indexes
WHERE object_id = OBJECT_ID(N'collect.query_store_data')
AND name = N'IX_query_store_data_id_lookup'
)
BEGIN
SET @index_sql = N'
CREATE NONCLUSTERED INDEX
IX_query_store_data_id_lookup
ON collect.query_store_data
(database_name, query_id, collection_time DESC)
WITH
(SORT_IN_TEMPDB = ON, DATA_COMPRESSION = PAGE' + @online_option + N');';
EXEC sys.sp_executesql @index_sql;
PRINT 'Created collect.query_store_data.IX_query_store_data_id_lookup index';
END;
Copy link
Copy Markdown
Owner Author

@erikdarlingdata erikdarlingdata Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Script is missing a trailing newline and a final GO. Every other upgrade script in this repo ends the last batch with a GO before EOF; the installer's batch splitter should still work without it, but match the house style from _template.sql.

Generated by Claude Code

1 change: 1 addition & 0 deletions upgrades/2.7.0-to-2.8.0/upgrade.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
01_add_query_lookup_indexes.sql
Loading