Skip to content

Integration tests against IRIS containers for release-preparation PRs - #359

Draft
isc-klu wants to merge 4 commits into
intersystems-community:masterfrom
isc-klu:integration-tests
Draft

isc-klu wants to merge 4 commits into
intersystems-community:masterfrom
isc-klu:integration-tests

Conversation

@isc-klu

@isc-klu isc-klu commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Replaces the placeholder npm test (a single assert.ok("All good"), run on every push) with an integration suite against two IRIS containers, run by a new prepare-release.yml workflow on PRs from prepare-* branches and on manual dispatch (Podman on the runner): ~1 min of container startup, then the cases.

Companion of intersystems-community/vscode-objectscript#1870. Both repos run the same 12-case matrix, each installing the other extension's Marketplace release; here the released ObjectScript extension is the API consumer.

What runs

src/test/cases.ts generates one .code-workspace per case into test-fixtures/.generated/; runTest.ts opens each in a downloaded VS Code with the released ObjectScript extension installed alongside, and src/test/suite/extension.test.ts runs the checks that apply. test-fixtures/README.md is the full matrix, the container layout, and how to run it locally.

Two containers back every case: iris (password only) and iris-anon (unauthenticated only), each with a 10-second /api/atelier session timeout so expired-session recovery meets a real 401.

Coverage

  • clientSide-os-hostobjectscript.conn host/port/credentials
  • clientSide-os-dockerobjectscript.conn.docker-compose port resolution (through Podman)
  • clientSide-smobjectscript.conn.server naming an intersystems.servers entry
  • serverSide-sm — an isfs:// folder on an entry

For every case, in a fresh user-data-dir: this extension's getServerSpec resolves the entry without prompting and a plaintext password reaches API consumers; makeRESTRequest lists the USER namespace as the Servers view does; the ObjectScript extension's asyncServerForUri reports the entry's host/port/credentials; a class written through the folder syncs to the server and back on delete iff active; a serverSide folder lists the namespace. Every check then repeats past the session timeout. A credential prompt fails the case: VS Code suppresses modal dialogs in tests and an input box would block to the mocha timeout.

Verified by mutation

Reverting #347 fails the password-reaches-consumers assertion; reverting #352 fails the anonymous listing checks.

Known failing on CI

The two clientSide-os-docker cases need the ObjectScript extension's Podman support, first shipped in a 3.8.6 beta but not yet on the Marketplace, so runTest.ts installs it with --pre-release and those cases will pass once that build is published. Until then they fail on CI; they pass locally against a Podman-capable build.

Not covered

  • The authentication provider's password prompt and secret storage paths (createSession, _isStillValid): they need typed input or seeded secret storage, and vscode.authentication.getSession from a test hits VS Code's consent dialog even for this extension.
  • Incorrect account detected #353: the fix was split between the two extensions and has no self-contained repro.
  • OAuth2 (OAuth2 VSCode #335): needs an identity provider.
  • The Servers view's tree items are not reachable from tests; the suite calls makeRESTRequest the way the view does.

Other changes

  • @vscode/test-electron 2.5 → 3.1 (2.x looks for a binary current VS Code no longer ships on macOS), plus skipLibCheck for its typings.
  • npm test removed from main.yml and prerelease.yml, since it now needs the containers.
  • A fresh --user-data-dir per case, and an assertion that the build under test (not the Marketplace release the ObjectScript extension pulls in as a dependency) is the one running.
  • .vscode/launch.json "Extension Tests" opens a generated case workspace.

🤖 Generated with Claude Code

Replace the placeholder suite with checks that every intersystems.servers
configuration connects without prompting, both through the extension's own
API/REST layer and through the released ObjectScript extension. Runs in a
new prepare-release.yml workflow on prepare-* PRs and on demand.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
isc-klu and others added 3 commits September 14, 2026 15:37
Run the IRIS containers under Podman and install the ObjectScript extension's
pre-release, whose Podman support the docker-compose case needs. Drop the
runner's Docker Compose so resolution goes through Podman. The docker-compose
cases fail until the ObjectScript Podman build reaches the Marketplace; the
post-timeout round-trip no longer re-verifies delete-sync, unreliable on the
current release.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant