Skip to content

fedora: implement Fedora Cloud image validation test suite - #4646

Open
Bala Konda Reddy (balakreddy) wants to merge 4 commits into
microsoft:mainfrom
balakreddy:fedora_ci_tests_4
Open

fedora: implement Fedora Cloud image validation test suite#4646
Bala Konda Reddy (balakreddy) wants to merge 4 commits into
microsoft:mainfrom
balakreddy:fedora_ci_tests_4

Conversation

@balakreddy

@balakreddy Bala Konda Reddy (balakreddy) commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Description

Expand the Fedora Cloud validation suite to cover 10 Fedora QA acceptance
test cases through LISA test methods.

Related Fedora QA Test Cases

Test Method Fedora QA Case
verify_fedora_edition_identification base_artwork_release_identification / base_edition_self_identification
verify_services_started base_services_start
verify_package_install_remove package_install_remove
verify_reboot_and_mounts base_reboot_unmount
verify_base_startup base_startup
verify_base_selinux base_selinux
verify_update_cli base_update_cli
verify_service_manipulation base_service_manipulation
verify_system_logging base_system_logging

Related Issue

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Refactoring
  • Documentation update

Checklist

  • Description is filled in above
  • No credentials, secrets, or internal details are included
  • Peer review requested (if not, add required peer reviewers after raising PR)
  • Tests executed and results posted below

Test Validation

Key Test Cases:

verify_fedora_edition_identification|verify_services_started|verify_package_install_remove|verify_reboot_and_mounts|verify_base_startup|verify_base_selinux|verify_update_cli|verify_service_manipulation|verify_system_logging

Impacted LISA Features:

Tested Azure Marketplace Images:

  • WestUS3/Fedora-5e266ba4-2250-406d-adad-5d73860d958f/Fedora-Cloud-44-x64/latest

Test Results

Image VM Size Result
WestUS3/Fedora-5e266ba4-2250-406d-adad-5d73860d958f/Fedora-Cloud-44-x64/latest
  |  Standard_D2ads_v5       | PASSED |

2026-08-12 23:45:48.233[140460735397632][INFO] lisa.RootRunner ________________________________________
2026-08-12 23:45:48.233[140460735397632][INFO] lisa.RootRunner FedoraCloudValidation.verify_fedora_edition_identification: PASSED
2026-08-12 23:45:48.233[140460735397632][INFO] lisa.RootRunner FedoraCloudValidation.verify_services_started: PASSED
2026-08-12 23:45:48.233[140460735397632][INFO] lisa.RootRunner FedoraCloudValidation.verify_package_install_remove: PASSED
2026-08-12 23:45:48.233[140460735397632][INFO] lisa.RootRunner FedoraCloudValidation.verify_reboot_and_mounts: PASSED
2026-08-12 23:45:48.233[140460735397632][INFO] lisa.RootRunner FedoraCloudValidation.verify_base_startup: PASSED
2026-08-12 23:45:48.233[140460735397632][INFO] lisa.RootRunner FedoraCloudValidation.verify_base_selinux: PASSED
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner FedoraCloudValidation.verify_update_cli: PASSED
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner FedoraCloudValidation.verify_service_manipulation: PASSED
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner FedoraCloudValidation.verify_system_logging: PASSED
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner test result summary
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner TOTAL : 9
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner QUEUED : 0
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner ASSIGNED : 0
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner RUNNING : 0
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner FAILED : 0
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner PASSED : 9
2026-08-12 23:45:48.234[140460735397632][INFO] lisa.RootRunner SKIPPED : 0
2026-08-12 23:45:48.238[140460735397632][INFO] lisa.notifier[Html] report: /home/lisa/runtime/log/20260812/20260812-233031-255/lisa.html
2026-08-12 23:45:48.240[140460735397632][INFO] lisa. completed in 916.989 sec

Add three test cases to FedoraCloudValidation suite:

- verify_startup_and_selinux: validates dmidecode, pciutils/lspci install
  cycle, and SELinux Enforcing mode with setenforce toggle
- verify_system_logging: validates journald boot log capture, audit
  entries, and journal integrity across reboot
- verify_user_management: validates full user lifecycle — useradd,
  chpasswd, usermod (shell + group), passwd -S lock/unlock, userdel

Signed-off-by: Bala Konda Reddy M <bala12352@gmail.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the Fedora Cloud validation test suite with additional coverage for (1) basic startup readiness + SELinux enforcement, (2) journald/audit logging behavior across reboot, and (3) end-to-end local user/group lifecycle operations. It also factors out the journal corruption scan into a helper used by multiple cases.

Changes:

  • Add _check_journal_corruption() helper and reuse it in reboot/logging validations.
  • Add three new Fedora validation test cases: startup+SELinux, system logging, and user management.
  • Expand tooling used in the suite by introducing the Usermod tool for group membership updates.
Suppressed comments (4)

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:405

  • Minor: use_new_environment=True is set, but the metadata description doesn’t explain why (unlike earlier cases in this file). Adding a short justification helps reviewers understand the extra environment cost.
        description="""
        Verify system logging via journalctl is working.

        Tests that journald captures boot logs, audit entries, and
        validates no filesystem corruption errors before/after reboot.
        """,
        priority=1,
        requirement=simple_requirement(supported_os=[Fedora]),
        use_new_environment=True,
    )

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:451

  • Minor: use_new_environment=True is set, but the metadata description doesn’t explain why (unlike earlier cases in this file). Adding a short justification makes the cost/need for a new environment explicit.
        description="""
        Verify user management operations (create, modify, delete).

        Tests useradd, usermod, chpasswd, account locking/unlocking,
        and userdel for complete user lifecycle management.
        """,
        priority=1,
        requirement=simple_requirement(supported_os=[Fedora]),
        use_new_environment=True,
    )

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:425

  • Nit: package_exists is defined on OperatingSystem, so the # type: ignore[attr-defined] suppression isn’t needed.
        if node.os.package_exists("rsyslog"):  # type: ignore[attr-defined]

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:506

  • Nit: node.os already defines install_packages, so the # type: ignore[attr-defined] suppression is unnecessary.
            node.os.install_packages("zsh")  # type: ignore[attr-defined]

Comment thread lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py
Comment thread lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py
Comment thread lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py
Comment thread lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py Outdated
Comment thread lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py Outdated
@balakreddy
Bala Konda Reddy (balakreddy) marked this pull request as draft August 11, 2026 23:39
Signed-off-by: Bala Konda Reddy M <bala12352@gmail.com>
Copilot AI review requested due to automatic review settings August 12, 2026 23:53
@balakreddy Bala Konda Reddy (balakreddy) changed the title fedora: add startup, logging, and user management validation tests fedora: implement Fedora Cloud image validation test suite Aug 12, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Suppressed comments (8)

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:131

  • Minor: The docstring says the test checks that SUPPORT_END is in the future, but the implementation only checks it if the field is present (it does not fail when SUPPORT_END is missing). Update the docstring to reflect the conditional behavior, or assert the field exists.
        - CPE_NAME includes :fedora:<VERSION_ID>
        - Installed fedora-release-common RPM version matches VERSION_ID
        - SUPPORT_END date is still in the future
        - PRETTY_NAME field is present

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:639

  • Nit: interval=2 is a test-behavior magic number; add an inline comment for maintainability (similar to the timeout comment).
                timeout=30,  # Allow rsyslog time to flush the message.
                interval=2,

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:79

  • Major: _check_journal_corruption reads the entire boot journal into memory and logs it at DEBUG via Tool.run (no_debug_log=False when out_file is None). On large journals this can bloat LISA logs and slow runs significantly. Prefer filtering server-side (journalctl + grep) and only returning matching lines.
        journalctl = node.tools[Journalctl]
        boot_logs = journalctl.first_n_logs_from_boot(boot_id=boot_id, no_of_lines=0)
        matches = [
            line
            for line in boot_logs.splitlines()

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:565

  • Minor: verify_service_manipulation disables chrony-wait.service but the finally block only restarts it; it doesn’t re-enable it. Restoring the original enablement state reduces risk of leaving the node in an unexpected configuration if the environment is reused or if post-test actions rely on chrony-wait.
        finally:
            systemctl.enable_service("chronyd.service")
            systemctl.start_service("chronyd.service")
            systemctl.start_service("chrony-wait.service")

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:95

  • Minor: _assert_chronyd_state validates enablement by comparing stdout from systemctl is-enabled, but doesn’t assert the exit code. systemctl is-enabled intentionally uses non-zero exit codes for non-enabled states, so stdout-only checks can hide failures (e.g., permission/unit errors). Using --quiet and asserting exit code is more reliable.
        enabled_result = node.execute(
            "systemctl is-enabled chronyd.service", no_error_log=True
        )
        assert_that(enabled_result.stdout.strip()).described_as(
            f"chronyd enablement must be correct {context}"
        ).is_equal_to("enabled" if expected_enabled else "disabled")

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:619

  • Nit: interval=2 is a test-behavior magic number; add an inline comment for maintainability (similar to the timeout comment).

This issue also appears on line 638 of the same file.

            timeout=30,  # Allow journald time to persist the message.
            interval=2,

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:622

  • Nit: OperatingSystem.package_exists() is defined on the base OS type, so the # type: ignore[attr-defined] is unnecessary and hides real typing issues.
        if node.os.package_exists("rsyslog"):  # type: ignore[attr-defined]

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:386

  • Major: The PR description says it adds three cases (verify_startup_and_selinux, verify_system_logging, verify_user_management) with specific validations (dmidecode/pciutils install cycle, setenforce toggle, full user lifecycle). This diff instead adds/renames different cases (verify_base_startup, verify_base_selinux, verify_update_cli, verify_service_manipulation) and does not add verify_user_management or the described SELinux toggle/package cycle. Please align the code with the PR description, or update the PR description to match what’s actually being added.
        """,
        priority=1,
        requirement=simple_requirement(supported_os=[Fedora]),
    )
    def verify_base_startup(self, node: Node) -> None:

@balakreddy
Bala Konda Reddy (balakreddy) marked this pull request as ready for review August 13, 2026 00:06
@balakreddy

Copy link
Copy Markdown
Contributor Author

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor note, but if it's not a generally useful helper for the object it seems fine as it is. This does look to line up with Fedora's documented acceptance testing.

Comment on lines +92 to +94
assert_that(enabled_result.stdout.strip()).described_as(
f"chronyd enablement must be correct {context}"
).is_equal_to("enabled" if expected_enabled else "disabled")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

systemctl will also exit 0 if it's enabled, non-zero if is not enabled, which is a bit nicer than string comparisons.

I see the service object has a is_service_running() function used below that returns a bool, it might be nice to add this as a is_service_enabled() helper.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, created new helper function is_service_enabled()

Signed-off-by: Bala Konda Reddy M <bala12352@gmail.com>
Copilot AI review requested due to automatic review settings August 14, 2026 18:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (6)

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:606

  • Major: journal_contains_test_message() ignores the journalctl exit code and doesn’t use sudo, so permission errors will look like “message not found” and cause flaky timeouts. Check exit_code and run journalctl with sudo=True.
        def journal_contains_test_message() -> bool:
            result = node.execute(
                f"journalctl --since '5 minutes ago' --no-pager -t {test_tag}",
                no_error_log=True,
            )

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:636

  • Minor: interval=2 is a test-behavior magic number; add an inline comment to document why this cadence is chosen for rsyslog flushing.
                interval=2,

lisa/base_tools/service.py:76

  • Major: Service.is_service_enabled() calls _internal_tool._is_service_enabled, but ServiceInternal (non-systemd path) doesn’t implement that method, so callers can hit an AttributeError on non-systemd distros. Add an explicit guard and raise a clear NotImplementedError instead of crashing.
    def is_service_enabled(self, name: str) -> bool:
        return self._internal_tool._is_service_enabled(name)  # type: ignore

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:235

  • Major: journalctl often requires elevated privileges (or membership in systemd-journal). Calling it without sudo=True can fail even when the system is healthy, causing a false test failure.

This issue also appears on line 602 of the same file.

        boot_journal = node.execute("journalctl -b --no-pager")

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:616

  • Minor: interval=2 is a test-behavior magic number; per project guidelines it should have an inline comment explaining why this polling cadence is appropriate.

This issue also appears on line 636 of the same file.

            interval=2,

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:373

  • Minor: The PR description says this suite covers 10 Fedora QA acceptance test cases, but this test suite currently defines 9 verify_* test methods (and the PR’s own Test Validation list also has 9). Please reconcile the count in the PR description/table (or add the missing 10th case).
    @TestCaseMetadata(

Comment thread lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py Outdated
Signed-off-by: Bala Konda Reddy M <bala12352@gmail.com>
Copilot AI review requested due to automatic review settings August 19, 2026 19:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (6)

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:578

  • use_new_environment=True is set without a brief justification comment. This test writes logs and performs a reboot, so it likely needs isolation, but please document the reason explicitly.
        use_new_environment=True,

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:493

  • use_new_environment=True is set but there’s no inline justification. Since this test disables/enables services and reboots, please document why a fresh VM is required (cost awareness + clarity).
        use_new_environment=True,

lisa/base_tools/service.py:77

  • Service.is_service_enabled() unconditionally calls self._internal_tool._is_service_enabled(...), but Service can fall back to ServiceInternal (non-systemd) where _is_service_enabled is not implemented. That would raise AttributeError at runtime on such distros.
    def is_service_enabled(self, name: str) -> bool:
        return self._internal_tool._is_service_enabled(name)  # type: ignore

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:439

  • use_new_environment=True is set but there’s no inline justification, while other tests in this suite explain why a fresh environment is required. Add a brief comment to document the cost/need tradeoff.

This issue also appears in the following locations of the same file:

  • line 493
  • line 578
        use_new_environment=True,

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:246

  • journalctl is invoked via node.execute() without sudo and bypasses the existing Journalctl tool (which defaults to sudo=True). On Fedora, non-root users commonly cannot read the full journal, so this can fail with permission errors.
        boot_journal = node.execute("journalctl -b --no-pager")
        assert_that(boot_journal.exit_code).described_as(
            "journalctl must read the current boot journal successfully"
        ).is_equal_to(0)
        assert_that(boot_journal.stdout.lower()).described_as(

lisa/microsoft/testsuites/fedora/fedora_cloud_validation.py:14

  • PR description says the suite covers 10 Fedora QA acceptance test cases, but this test suite currently defines 9 verify_... test methods and the posted test run summary also shows 9 total. Please either add the missing 10th case or update the PR description/table to match the implemented scope.
from uuid import uuid4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants