fedora: implement Fedora Cloud image validation test suite - #4646
fedora: implement Fedora Cloud image validation test suite#4646Bala Konda Reddy (balakreddy) wants to merge 4 commits into
Conversation
Add three test cases to FedoraCloudValidation suite: - verify_startup_and_selinux: validates dmidecode, pciutils/lspci install cycle, and SELinux Enforcing mode with setenforce toggle - verify_system_logging: validates journald boot log capture, audit entries, and journal integrity across reboot - verify_user_management: validates full user lifecycle — useradd, chpasswd, usermod (shell + group), passwd -S lock/unlock, userdel Signed-off-by: Bala Konda Reddy M <bala12352@gmail.com>
There was a problem hiding this comment.
Pull request overview
This PR extends the Fedora Cloud validation test suite with additional coverage for (1) basic startup readiness + SELinux enforcement, (2) journald/audit logging behavior across reboot, and (3) end-to-end local user/group lifecycle operations. It also factors out the journal corruption scan into a helper used by multiple cases.
Changes:
- Add
_check_journal_corruption()helper and reuse it in reboot/logging validations. - Add three new Fedora validation test cases: startup+SELinux, system logging, and user management.
- Expand tooling used in the suite by introducing the
Usermodtool for group membership updates.
Suppressed comments (4)
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:405
- Minor:
use_new_environment=Trueis set, but the metadata description doesn’t explain why (unlike earlier cases in this file). Adding a short justification helps reviewers understand the extra environment cost.
description="""
Verify system logging via journalctl is working.
Tests that journald captures boot logs, audit entries, and
validates no filesystem corruption errors before/after reboot.
""",
priority=1,
requirement=simple_requirement(supported_os=[Fedora]),
use_new_environment=True,
)
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:451
- Minor:
use_new_environment=Trueis set, but the metadata description doesn’t explain why (unlike earlier cases in this file). Adding a short justification makes the cost/need for a new environment explicit.
description="""
Verify user management operations (create, modify, delete).
Tests useradd, usermod, chpasswd, account locking/unlocking,
and userdel for complete user lifecycle management.
""",
priority=1,
requirement=simple_requirement(supported_os=[Fedora]),
use_new_environment=True,
)
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:425
- Nit:
package_existsis defined onOperatingSystem, so the# type: ignore[attr-defined]suppression isn’t needed.
if node.os.package_exists("rsyslog"): # type: ignore[attr-defined]
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:506
- Nit:
node.osalready definesinstall_packages, so the# type: ignore[attr-defined]suppression is unnecessary.
node.os.install_packages("zsh") # type: ignore[attr-defined]
Signed-off-by: Bala Konda Reddy M <bala12352@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.
Suppressed comments (8)
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:131
- Minor: The docstring says the test checks that SUPPORT_END is in the future, but the implementation only checks it if the field is present (it does not fail when SUPPORT_END is missing). Update the docstring to reflect the conditional behavior, or assert the field exists.
- CPE_NAME includes :fedora:<VERSION_ID>
- Installed fedora-release-common RPM version matches VERSION_ID
- SUPPORT_END date is still in the future
- PRETTY_NAME field is present
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:639
- Nit: interval=2 is a test-behavior magic number; add an inline comment for maintainability (similar to the timeout comment).
timeout=30, # Allow rsyslog time to flush the message.
interval=2,
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:79
- Major: _check_journal_corruption reads the entire boot journal into memory and logs it at DEBUG via Tool.run (no_debug_log=False when out_file is None). On large journals this can bloat LISA logs and slow runs significantly. Prefer filtering server-side (journalctl + grep) and only returning matching lines.
journalctl = node.tools[Journalctl]
boot_logs = journalctl.first_n_logs_from_boot(boot_id=boot_id, no_of_lines=0)
matches = [
line
for line in boot_logs.splitlines()
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:565
- Minor: verify_service_manipulation disables chrony-wait.service but the finally block only restarts it; it doesn’t re-enable it. Restoring the original enablement state reduces risk of leaving the node in an unexpected configuration if the environment is reused or if post-test actions rely on chrony-wait.
finally:
systemctl.enable_service("chronyd.service")
systemctl.start_service("chronyd.service")
systemctl.start_service("chrony-wait.service")
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:95
- Minor: _assert_chronyd_state validates enablement by comparing stdout from
systemctl is-enabled, but doesn’t assert the exit code.systemctl is-enabledintentionally uses non-zero exit codes for non-enabled states, so stdout-only checks can hide failures (e.g., permission/unit errors). Using--quietand asserting exit code is more reliable.
enabled_result = node.execute(
"systemctl is-enabled chronyd.service", no_error_log=True
)
assert_that(enabled_result.stdout.strip()).described_as(
f"chronyd enablement must be correct {context}"
).is_equal_to("enabled" if expected_enabled else "disabled")
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:619
- Nit: interval=2 is a test-behavior magic number; add an inline comment for maintainability (similar to the timeout comment).
This issue also appears on line 638 of the same file.
timeout=30, # Allow journald time to persist the message.
interval=2,
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:622
- Nit:
OperatingSystem.package_exists()is defined on the base OS type, so the# type: ignore[attr-defined]is unnecessary and hides real typing issues.
if node.os.package_exists("rsyslog"): # type: ignore[attr-defined]
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:386
- Major: The PR description says it adds three cases (verify_startup_and_selinux, verify_system_logging, verify_user_management) with specific validations (dmidecode/pciutils install cycle, setenforce toggle, full user lifecycle). This diff instead adds/renames different cases (verify_base_startup, verify_base_selinux, verify_update_cli, verify_service_manipulation) and does not add verify_user_management or the described SELinux toggle/package cycle. Please align the code with the PR description, or update the PR description to match what’s actually being added.
""",
priority=1,
requirement=simple_requirement(supported_os=[Fedora]),
)
def verify_base_startup(self, node: Node) -> None:
Jeremy Cline (jeremycline)
left a comment
There was a problem hiding this comment.
One minor note, but if it's not a generally useful helper for the object it seems fine as it is. This does look to line up with Fedora's documented acceptance testing.
| assert_that(enabled_result.stdout.strip()).described_as( | ||
| f"chronyd enablement must be correct {context}" | ||
| ).is_equal_to("enabled" if expected_enabled else "disabled") |
There was a problem hiding this comment.
systemctl will also exit 0 if it's enabled, non-zero if is not enabled, which is a bit nicer than string comparisons.
I see the service object has a is_service_running() function used below that returns a bool, it might be nice to add this as a is_service_enabled() helper.
There was a problem hiding this comment.
Done, created new helper function is_service_enabled()
Signed-off-by: Bala Konda Reddy M <bala12352@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.
Suppressed comments (6)
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:606
- Major:
journal_contains_test_message()ignores thejournalctlexit code and doesn’t use sudo, so permission errors will look like “message not found” and cause flaky timeouts. Checkexit_codeand runjournalctlwithsudo=True.
def journal_contains_test_message() -> bool:
result = node.execute(
f"journalctl --since '5 minutes ago' --no-pager -t {test_tag}",
no_error_log=True,
)
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:636
- Minor:
interval=2is a test-behavior magic number; add an inline comment to document why this cadence is chosen for rsyslog flushing.
interval=2,
lisa/base_tools/service.py:76
- Major:
Service.is_service_enabled()calls_internal_tool._is_service_enabled, butServiceInternal(non-systemd path) doesn’t implement that method, so callers can hit anAttributeErroron non-systemd distros. Add an explicit guard and raise a clearNotImplementedErrorinstead of crashing.
def is_service_enabled(self, name: str) -> bool:
return self._internal_tool._is_service_enabled(name) # type: ignore
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:235
- Major:
journalctloften requires elevated privileges (or membership insystemd-journal). Calling it withoutsudo=Truecan fail even when the system is healthy, causing a false test failure.
This issue also appears on line 602 of the same file.
boot_journal = node.execute("journalctl -b --no-pager")
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:616
- Minor:
interval=2is a test-behavior magic number; per project guidelines it should have an inline comment explaining why this polling cadence is appropriate.
This issue also appears on line 636 of the same file.
interval=2,
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:373
- Minor: The PR description says this suite covers 10 Fedora QA acceptance test cases, but this test suite currently defines 9
verify_*test methods (and the PR’s own Test Validation list also has 9). Please reconcile the count in the PR description/table (or add the missing 10th case).
@TestCaseMetadata(
Signed-off-by: Bala Konda Reddy M <bala12352@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.
Suppressed comments (6)
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:578
use_new_environment=Trueis set without a brief justification comment. This test writes logs and performs a reboot, so it likely needs isolation, but please document the reason explicitly.
use_new_environment=True,
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:493
use_new_environment=Trueis set but there’s no inline justification. Since this test disables/enables services and reboots, please document why a fresh VM is required (cost awareness + clarity).
use_new_environment=True,
lisa/base_tools/service.py:77
Service.is_service_enabled()unconditionally callsself._internal_tool._is_service_enabled(...), butServicecan fall back toServiceInternal(non-systemd) where_is_service_enabledis not implemented. That would raiseAttributeErrorat runtime on such distros.
def is_service_enabled(self, name: str) -> bool:
return self._internal_tool._is_service_enabled(name) # type: ignore
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:439
use_new_environment=Trueis set but there’s no inline justification, while other tests in this suite explain why a fresh environment is required. Add a brief comment to document the cost/need tradeoff.
This issue also appears in the following locations of the same file:
- line 493
- line 578
use_new_environment=True,
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:246
journalctlis invoked vianode.execute()withoutsudoand bypasses the existingJournalctltool (which defaults tosudo=True). On Fedora, non-root users commonly cannot read the full journal, so this can fail with permission errors.
boot_journal = node.execute("journalctl -b --no-pager")
assert_that(boot_journal.exit_code).described_as(
"journalctl must read the current boot journal successfully"
).is_equal_to(0)
assert_that(boot_journal.stdout.lower()).described_as(
lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:14
- PR description says the suite covers 10 Fedora QA acceptance test cases, but this test suite currently defines 9
verify_...test methods and the posted test run summary also shows 9 total. Please either add the missing 10th case or update the PR description/table to match the implemented scope.
from uuid import uuid4
Description
Expand the Fedora Cloud validation suite to cover 10 Fedora QA acceptance
test cases through LISA test methods.
Related Fedora QA Test Cases
verify_fedora_edition_identificationverify_services_startedverify_package_install_removeverify_reboot_and_mountsverify_base_startupverify_base_selinuxverify_update_cliverify_service_manipulationverify_system_loggingRelated Issue
Type of Change
Checklist
Test Validation
Key Test Cases:
verify_fedora_edition_identification|verify_services_started|verify_package_install_remove|verify_reboot_and_mounts|verify_base_startup|verify_base_selinux|verify_update_cli|verify_service_manipulation|verify_system_logging
Impacted LISA Features:
Tested Azure Marketplace Images:
Test Results
2026-08-12 23:45:48.233[140460735397632][INFO] lisa.RootRunner ________________________________________
2026-08-12 23:45:48.233[140460735397632][INFO] lisa.RootRunner FedoraCloudValidation.verify_fedora_edition_identification: PASSED
2026-08-12 23:45:48.233[140460735397632][INFO] lisa.RootRunner FedoraCloudValidation.verify_services_started: PASSED
2026-08-12 23:45:48.233[140460735397632][INFO] lisa.RootRunner FedoraCloudValidation.verify_package_install_remove: PASSED
2026-08-12 23:45:48.233[140460735397632][INFO] lisa.RootRunner FedoraCloudValidation.verify_reboot_and_mounts: PASSED
2026-08-12 23:45:48.233[140460735397632][INFO] lisa.RootRunner FedoraCloudValidation.verify_base_startup: PASSED
2026-08-12 23:45:48.233[140460735397632][INFO] lisa.RootRunner FedoraCloudValidation.verify_base_selinux: PASSED
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner FedoraCloudValidation.verify_update_cli: PASSED
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner FedoraCloudValidation.verify_service_manipulation: PASSED
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner FedoraCloudValidation.verify_system_logging: PASSED
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner test result summary
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner TOTAL : 9
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner QUEUED : 0
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner ASSIGNED : 0
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner RUNNING : 0
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner FAILED : 0
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner PASSED : 9
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner SKIPPED : 0
2026-08-12 23:45:48.238[140460735397632][INFO] lisa.notifier[Html] report: /home/lisa/runtime/log/20260812/20260812-233031-255/lisa.html
2026-08-12 23:45:48.240[140460735397632][INFO] lisa. completed in 916.989 sec