Skip to content

chore: increase test coverage and fix temp directory use - #639

Merged
fzipi merged 1 commit into
coreruleset:mainfrom
theseion:increase-test-coverage
May 25, 2026
Merged

chore: increase test coverage and fix temp directory use#639
fzipi merged 1 commit into
coreruleset:mainfrom
theseion:increase-test-coverage

Conversation

@theseion

@theseion theseion commented May 23, 2026

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • Tests

    • Standardized per-test temporary-directory handling across many suites for more reliable file-based tests and consistent cleanup.
    • Added new unit tests for log ID-matching behavior to improve coverage and validate matching semantics.
    • Strengthened test assertions to surface file creation and parsing errors.
  • Chores

    • Simplified and standardized test helper utilities for creating temporary files and unified test setup.

Review Change Stack

@theseion
theseion requested a review from fzipi May 23, 2026 15:17
@coderabbitai

coderabbitai Bot commented May 23, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@theseion, we couldn't start this review because you've used your available PR reviews for now.

Your plan currently allows 2 reviews/hour. Refill in 9 minutes and 42 seconds.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more review capacity refills, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than trial, open-source, and free plans. In all cases, review capacity refills continuously over time.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 840924a6-52b8-4f47-9d63-18c9fe6668ec

📥 Commits

Reviewing files that changed from the base of the PR and between 9b8fbff and b4432b3.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (15)
  • cmd/root_test.go
  • cmd/run/run_test.go
  • config/config_test.go
  • internal/quantitative/runner_test.go
  • runner/check_base_test.go
  • runner/check_error_test.go
  • runner/check_logs_test.go
  • runner/check_response_test.go
  • runner/check_status_test.go
  • runner/run_test.go
  • test/files_test.go
  • utils/tests.go
  • utils/tests_test.go
  • waflog/read_test.go
  • waflog/waflog_test.go
📝 Walkthrough

Walkthrough

All tests now use explicit per-test tempDir fields initialized in suite setup. utils.CreateTempFileWithContent/CreateTempFile require and validate a provided directory. Test code across command, config, runner, files, and waflog suites was updated to use the suite tempDir; waflog adds two new tests.

Changes

Test Infrastructure Refactoring: Centralized Temporary Directories

Layer / File(s) Summary
Core utility function refactoring
utils/tests.go, utils/tests_test.go
CreateTempFileWithContent and CreateTempFile now require an explicit valid dir, validate it with os.Stat, create temp files inside that dir, and tests updated to pass suite-scoped temp directories.
Command test suite centralization
cmd/root_test.go, cmd/run/run_test.go, runner/run_test.go
Root/run command suites add tempDir fields initialized in setup and reuse them for config, overrides, mock YAML, and failure-waf-logs temp files instead of creating per-call temp dirs.
Config and quantitative suite centralization
config/config_test.go, internal/quantitative/runner_test.go
Config suite adds tempDir; quantitative runner test suite renames dirtempDir and updates Params, getter destinations, and corpus file paths to use the suite temp directory.
Runner check test suites centralization
runner/check_base_test.go, runner/check_error_test.go, runner/check_logs_test.go, runner/check_response_test.go, runner/check_status_test.go
Multiple runner check suites add tempDir fields and create ModSecurity/log fixtures and test YAML files inside the suite temp directory via the updated utils functions.
Files and WAF log test suite centralization with new coverage
test/files_test.go, waflog/read_test.go
Files suite uses tempDir for YAML fixtures. WAF log suite gains tempDir, writes full log content into suite temp files, removes manual TearDownSuite cleanup, and adds TestFTWLogLines_ContainsAllIds and TestFTWLogLines_ContainsAnyId.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested reviewers

  • fzipi

🐰 I hopped through temp dirs, neat and spry,
Suite-scoped files now cozy where they lie,
No more stray dirs scattered far and wild,
Tests stay tidy — says this little rabbit-child.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 18.18% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main changes in the PR, which involve refactoring temporary directory usage across test suites and adding new test coverage.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@config/config_test.go`:
- Line 71: The test currently ignores the error returned by
utils.CreateTempFileWithContent when assigning s.filename; update both call
sites (the one setting s.filename and the one on the later line) to capture the
error (e.g., filename, err := utils.CreateTempFileWithContent(...)) and fail the
test immediately on error (use s.T().Fatalf or require.NoError/suite assertions)
so a failed temp-file creation reports the real error instead of causing a
misleading failure later.

In `@test/files_test.go`:
- Line 69: The test currently ignores the error returned by
utils.CreateTempFileWithContent by using the blank identifier; update the call
to capture the error (e.g., filename, err :=
utils.CreateTempFileWithContent(...)) and add a fail-fast assertion
s.Require().NoError(err) immediately after to stop the test if temp file
creation fails; apply the same change for the other CreateTempFileWithContent
invocation in these tests.

In `@utils/tests.go`:
- Line 14: Update the GoDoc for CreateTempFileWithContent to state that the dir
argument must be a valid, non-empty directory and that the function no longer
falls back to os.TempDir(); change any phrasing implying an automatic fallback
to explicitly document the explicit-dir contract, and apply the same wording
update to the other temp helper functions in this file so their comments
accurately reflect the new requirement.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 956c946c-9533-416f-ab62-ea0658bc0502

📥 Commits

Reviewing files that changed from the base of the PR and between b72bd67 and 417b709.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (14)
  • cmd/root_test.go
  • cmd/run/run_test.go
  • config/config_test.go
  • internal/quantitative/runner_test.go
  • runner/check_base_test.go
  • runner/check_error_test.go
  • runner/check_logs_test.go
  • runner/check_response_test.go
  • runner/check_status_test.go
  • runner/run_test.go
  • test/files_test.go
  • utils/tests.go
  • utils/tests_test.go
  • waflog/read_test.go

Comment thread config/config_test.go Outdated
Comment thread test/files_test.go Outdated
Comment thread utils/tests.go
@theseion
theseion force-pushed the increase-test-coverage branch from 417b709 to 3916d4e Compare May 24, 2026 06:09

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
waflog/read_test.go (1)

159-167: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Add missing FTWLogLines cleanup in this test path.

This test creates ll via NewFTWLogLines but does not register ll.Cleanup() (unlike adjacent tests), which can leak handles and make tests flaky over time.

Suggested fix
 ll, err := NewFTWLogLines(runnerConfig)
 s.Require().NoError(err)
+s.T().Cleanup(func() { _ = ll.Cleanup() })
 ll.WithStartMarker(bytes.ToLower([]byte(startMarkerLine)))
 ll.WithEndMarker(bytes.ToLower([]byte(endMarkerLine)))
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@waflog/read_test.go` around lines 159 - 167, The test instantiates
FTWLogLines via NewFTWLogLines (assigned to ll) but never calls ll.Cleanup(),
risking resource leaks; after creating ll and asserting NoError (the
NewFTWLogLines call), add a deferred cleanup by calling ll.Cleanup() (e.g.,
defer ll.Cleanup()) so FTWLogLines resources are released for this test path.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@waflog/read_test.go`:
- Around line 159-167: The test instantiates FTWLogLines via NewFTWLogLines
(assigned to ll) but never calls ll.Cleanup(), risking resource leaks; after
creating ll and asserting NoError (the NewFTWLogLines call), add a deferred
cleanup by calling ll.Cleanup() (e.g., defer ll.Cleanup()) so FTWLogLines
resources are released for this test path.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: d3f21cd4-38f2-44fe-99a4-db2d3573504b

📥 Commits

Reviewing files that changed from the base of the PR and between 417b709 and 3916d4e.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (14)
  • cmd/root_test.go
  • cmd/run/run_test.go
  • config/config_test.go
  • internal/quantitative/runner_test.go
  • runner/check_base_test.go
  • runner/check_error_test.go
  • runner/check_logs_test.go
  • runner/check_response_test.go
  • runner/check_status_test.go
  • runner/run_test.go
  • test/files_test.go
  • utils/tests.go
  • utils/tests_test.go
  • waflog/read_test.go
✅ Files skipped from review due to trivial changes (1)
  • test/files_test.go

@theseion
theseion force-pushed the increase-test-coverage branch from 3916d4e to 9b8fbff Compare May 24, 2026 06:16

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
waflog/read_test.go (1)

159-167: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Add cleanup for FTWLogLines in this test case.

After Line 165, this test creates ll but does not register ll.Cleanup(), unlike adjacent updated tests. That can leak file descriptors across the suite.

Suggested fix
 	ll, err := NewFTWLogLines(runnerConfig)
 	s.Require().NoError(err)
+	s.T().Cleanup(func() { _ = ll.Cleanup() })
 	ll.WithStartMarker(bytes.ToLower([]byte(startMarkerLine)))
 	ll.WithEndMarker(bytes.ToLower([]byte(endMarkerLine)))
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@waflog/read_test.go` around lines 159 - 167, The test creates an FTWLogLines
instance (ll := NewFTWLogLines(...)) but never registers its cleanup, risking
leaked descriptors; after successfully creating ll (and after
s.Require().NoError(err)), add a defer ll.Cleanup() (or
s.T().Cleanup(ll.Cleanup)) so the FTWLogLines.Cleanup() method is invoked when
the test finishes.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@waflog/read_test.go`:
- Around line 159-167: The test creates an FTWLogLines instance (ll :=
NewFTWLogLines(...)) but never registers its cleanup, risking leaked
descriptors; after successfully creating ll (and after
s.Require().NoError(err)), add a defer ll.Cleanup() (or
s.T().Cleanup(ll.Cleanup)) so the FTWLogLines.Cleanup() method is invoked when
the test finishes.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 89fb2a07-0457-4ab9-9207-b18a586aa353

📥 Commits

Reviewing files that changed from the base of the PR and between 3916d4e and 9b8fbff.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (14)
  • cmd/root_test.go
  • cmd/run/run_test.go
  • config/config_test.go
  • internal/quantitative/runner_test.go
  • runner/check_base_test.go
  • runner/check_error_test.go
  • runner/check_logs_test.go
  • runner/check_response_test.go
  • runner/check_status_test.go
  • runner/run_test.go
  • test/files_test.go
  • utils/tests.go
  • utils/tests_test.go
  • waflog/read_test.go
✅ Files skipped from review due to trivial changes (2)
  • runner/check_status_test.go
  • internal/quantitative/runner_test.go

@theseion
theseion force-pushed the increase-test-coverage branch from 9b8fbff to b4432b3 Compare May 24, 2026 06:29

@M4tteoP M4tteoP left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@fzipi
fzipi merged commit 9e419b4 into coreruleset:main May 25, 2026
5 checks passed
@theseion
theseion deleted the increase-test-coverage branch May 25, 2026 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants