OSPS-QA.yaml

groups:
  - id: QA
    title: Quality
    description: |
      Quality focuses on the processes and
      practices used to ensure the quality and
      reliability of the project's source code and
      software assets. These controls help ensure
      that the project's source code is well
      maintained, secure, and reliable, reducing the
      risk of defects or vulnerabilities in the
      software.
controls:
  - id: OSPS-QA-01
    title: |
      Publish Source Code and Change History
    objective: |
      Enable users to access and review the project's source code and history,
      promoting transparency and collaboration within the project community.
    group: QA
    assessment-requirements:
      - id: OSPS-QA-01.01
        text: |
          While active, the project's source code repository MUST be publicly
          readable at a static URL.
        applicability:
          - maturity-1
          - maturity-2
          - maturity-3
        recommendation: |
          Use a common VCS such as GitHub, GitLab, or Bitbucket. Ensure the
          repository is publicly readable. Avoid duplication or mirroring of
          repositories unless highly visible documentation clarifies the primary
          source. Avoid frequent changes to the repository that would impact the
          repository URL. Ensure the repository is public.
      - id: OSPS-QA-01.02
        text: |
          The version control system MUST contain a publicly readable record of
          all changes made, who made the changes, and when the changes were
          made.
        applicability:
          - maturity-1
          - maturity-2
          - maturity-3
        recommendation: |
          Use a common VCS such as GitHub, GitLab, or Bitbucket to maintain a
          publicly readable commit history. Avoid squashing or rewriting commits
          in a way that would obscure the author of any commits.

  - id: OSPS-QA-02
    title: |
      Publish Software Dependencies
    objective: |
      Provide transparency and accountability for the project's dependencies
      while enabling users and contributors to understand the software's direct
      dependencies.
    group: QA
    assessment-requirements:
      - id: OSPS-QA-02.01
        text: |
          When the package management system supports it, the source code
          repository MUST contain a dependency list that accounts for the direct
          language dependencies.
        applicability:
          - maturity-1
          - maturity-2
          - maturity-3
        recommendation: |
          This may take the form of a package manager or language dependency file
          that enumerates all direct dependencies such as package.json, Gemfile,
          or go.mod.
      - id: OSPS-QA-02.02
        text: |
          When the project has made a release, all compiled released software
          assets MUST be delivered with a software bill of materials.
        applicability:
          - maturity-3
        recommendation: |
          It is recommended to auto-generate SBOMs at build time using a tool
          that has been vetted for accuracy. This enables users to ingest this
          data in a standardized approach alongside other projects in their
          environment.

  - id: OSPS-QA-03
    title: |
      Address Pass/Fail Checks Before Accepting Changes
    objective: |
      Ensure that the project's approvers do not become accustomed to tolerating
      failing status checks, even if arbitrary, because it increases the risk of
      overlooking security vulnerabilities or defects identified by automated
      checks.
    group: QA
    assessment-requirements:
      - id: OSPS-QA-03.01
        text: |
          When a commit is made to the primary branch, any automated status
          checks for commits MUST pass or be manually bypassed.
        applicability:
          - maturity-2
          - maturity-3
        recommendation: |
          Configure the project's version control system to require that all
          automated status checks pass or require manual acknowledgement before a
          commit can be merged into the primary branch. It is recommended that
          any optional status checks are NOT configured as a pass or fail
          requirement that approvers may be tempted to bypass.

  - id: OSPS-QA-04
    title: |
      Enforce Security Requirements on All Codebases
    objective: |
      Ensure that all codebases produced by the project are well
      documented and held to the same security standard.
    group: QA
    assessment-requirements:
      - id: OSPS-QA-04.01
        text: |
          Projects with multiple repositories MUST document a list of codebases
          that are part of the project.
        applicability:
          - maturity-1
          - maturity-2
          - maturity-3
        recommendation: |
          Document any additional subproject code repositories produced by the
          project and compiled into a release. This documentation should include
          the status and intent of the respective codebase.
      - id: OSPS-QA-04.02
        text: |
          When the project has made a release comprising multiple source code
          repositories, all subprojects MUST enforce security requirements that
          are as strict or stricter than the primary codebase.
        applicability:
          - maturity-3
        recommendation: |
          Any additional subproject code repositories produced by the project
          and compiled into a release must enforce security requirements as
          applicable to the status and intent of the respective codebase.
          In addition to following the corresponding OSPS Baseline requirements,
          this may include requiring a security review, ensuring that it is
          free of vulnerabilities, and ensuring that it is free of known
          security issues.

  - id: OSPS-QA-05
    title: |
      Prevent Executables in the Codebase
    objective: |
      Reduce the risk of including generated executable artifacts in the
      project's version control system, ensuring that only source code and
      necessary files are stored in the repository.
    group: QA
    assessment-requirements:
      - id: OSPS-QA-05.01
        text: |
          While active, the version control system MUST NOT contain generated
          executable artifacts.
        applicability:
          - maturity-1
          - maturity-2
          - maturity-3
        recommendation: |
          Remove generated executable artifacts in the project's version control
          system. It is recommended that any scenario where a generated
          executable artifact appears critical to a process such as testing, it
          should be instead be generated at build time or stored separately and
          fetched during a specific well-documented pipeline step.
      - id: OSPS-QA-05.02
        text: |
          While active, the version control system MUST NOT contain unreviewable
          binary artifacts.
        applicability:
          - maturity-1
          - maturity-2
          - maturity-3
        recommendation: |
          Do not add any unreviewable binary artifacts to the project's version
          control system. This includes executable application binaries, library
          files, and similar artifacts. It does not include assets such as
          graphical images, sound or music files, and similar content typically
          stored in a binary format.

  - id: OSPS-QA-06
    title: |
      Use Automated Testing in CI/CD Pipelines
    objective: |
      Ensure that the project uses at least one automated test suite for the
      source code repository and clearly documents when and how tests are run.
    group: QA
    assessment-requirements:
      - id: OSPS-QA-06.01
        text: |
          Prior to a commit being accepted, the project's CI/CD pipelines MUST
          run at least one automated test suite to ensure the changes meet
          expectations.
        applicability:
          - maturity-2
          - maturity-3
        recommendation: | # give examples
          Automated tests should be run prior to every merge into the primary
          branch. The test suite should be run in a CI/CD pipeline and the
          results should be visible to all contributors. The test suite should
          be run in a consistent environment and should be run in a way that
          allows contributors to run the tests locally.
          Examples of test suites include unit tests, integration tests, and
          end-to-end tests.
      - id: OSPS-QA-06.02
        text: |
          While active, project's documentation MUST clearly document when and
          how tests are run.
        applicability:
          - maturity-3
        recommendation: |
          Add a section to the contributing documentation that explains how to
          run the tests locally and how to run the tests in the CI/CD pipeline.
          The documentation should explain what the tests are testing and how to
          interpret the results.
      - id: OSPS-QA-06.03
        text: |
          While active, the project's documentation MUST include a policy that
          all major changes to the software produced by the project should add
          or update tests of the functionality in an automated test suite.
        applicability:
          - maturity-3
        recommendation: |
          Add a section to the contributing documentation that explains the
          policy for adding or updating tests. The policy should explain what
          constitutes a major change and what tests should be added or updated.

  - id: OSPS-QA-07
    title: |
      Require Merge Approvals
    objective: |
      Ensure that the project's version control system requires at least one
      non-author human approval of changes before merging into the release or
      primary branch.
    group: QA
    assessment-requirements:
      - id: OSPS-QA-07.01
        text: |
          When a commit is made to the primary branch, the project's version
          control system MUST require at least one non-author human approval of the
          changes before merging.
        applicability:
          - maturity-3
        recommendation: |
          Configure the project's version control system to require at least one
          non-author human approval of changes before merging into the release or
          primary branch. This can be achieved by requiring a pull request to be
          reviewed and approved by at least one other collaborator before it can
          be merged.