As visible at e.g. https://github.com/tc39/test262/actions/runs/33647550583?pr=5125#artifacts , results-$engineName.json files have a structure like
Array<{
file: FilesystemPath;
relative: FilesystemPath;
attrs: FrontmatterRecord;
copyright: string;
contents: FileContents;
scenario: "default" | "strict mode";
compiled: FileContents;
insertionIndex: number;
rawResult: { stderr: string, stdout: string, error: null | { name: string, message: string | null, stack: object[] } };
result: { pass: true } | { pass: false, message: string };
}>
I can imagine this being useful sometimes, but it's overwhelming to deal with manually, and honestly seems rather wasteful anyway. It's also the only way to see what's going on, since the "Run changed tests" GitHub Actions workflow step redirects all output into the JSON file that ultimately gets uploaded.
In the common case, we probably don't need anything more than { file, result } pairs, ideally visible directly in CI output, and if we want more detail then it seems appropriate to make that opt-in via e.g. a label or comment on the PR.
As visible at e.g. https://github.com/tc39/test262/actions/runs/33647550583?pr=5125#artifacts , results-$engineName.json files have a structure like
I can imagine this being useful sometimes, but it's overwhelming to deal with manually, and honestly seems rather wasteful anyway. It's also the only way to see what's going on, since the "Run changed tests" GitHub Actions workflow step redirects all output into the JSON file that ultimately gets uploaded.
In the common case, we probably don't need anything more than { file, result } pairs, ideally visible directly in CI output, and if we want more detail then it seems appropriate to make that opt-in via e.g. a label or comment on the PR.