feat: add extensible test artifact API #8987
Conversation
✅ Deploy Preview for vitest-dev ready!Built without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
macarie
left a comment
macarie
left a comment
There was a problem hiding this comment.
Didn't change/add any new docs, would like to get a first review and check that everything is okay first.
There was a problem hiding this comment.
This test is still very much in-progress, still have to figure out a couple of things.
| }) | ||
| }) | ||
|
|
||
| describe('reporters', () => { |
There was a problem hiding this comment.
Should these built-in reporters have any general handling for artifacts?
Personally, I don't think they should, but it should be an artifact-by-artifact decision.
There was a problem hiding this comment.
Modified this only to check that annotations are not added to artifacts.
macarie
force-pushed
the
feat/record-artifacts
branch
2 times, most recently
from
3c73e71 to
7e3aaf7
Compare
docs/api/advanced/artifacts.md
Outdated
| title: Test Artifacts | ||
| --- | ||
|
|
||
| # Test Artifacts <Badge type="danger">advanced</Badge> <Badge type="warning">experimental</Badge> |
There was a problem hiding this comment.
We have <Experimental /> component now 😄 I think we should also add Advanced
There was a problem hiding this comment.
We should also have a version here. I wonder if experimental features should have a warning color in the Version component though 🤔
docs/api/advanced/reporters.md
Outdated
| - [`onHookEnd(beforeAll)`](#onhookend) | ||
| - [`onTestCaseReady`](#ontestcaseready) | ||
| - [`onTestAnnotate`](#ontestannotate) <Version>3.2.0</Version> | ||
| - [`onTestCaseArtifactRecord`](#ontestcaseartifactrecord) <Version>4.0.9</Version> |
There was a problem hiding this comment.
Can we change the Version color to be warning when it's experimental? In 4.1 we can revert it to normal and switch to 4.1
docs/.vitepress/config.ts
Outdated
| link: '/api/advanced/metadata', | ||
| }, | ||
| { | ||
| text: 'Artifacts', |
There was a problem hiding this comment.
Maybe call it TestArtifact like the interface
|
Note: update |
macarie
force-pushed
the
feat/record-artifacts
branch
from
bbf7a72 to
7cc3b57
Compare
macarie
force-pushed
the
feat/record-artifacts
branch
from
7cc3b57 to
46c5db1
Compare
2 participants
Description
A spin-off of #8813 and third attempt at creating an extensible API for handling test artifacts produced by Vitest and third-parties.
After talking more with @sheremet-va about the issues encountered while developing #8813 and the current API, we came up with this solution, which should make it easier for developers building on top of Vitest, as well as improve internal artifact handling.
The main features of this version are:
TestArtifactRegistrymodule augmentationTestArtifactBaseinterface as foundation for all artifactsTestArtifactunion type containing all artifact types that can be produced (including custom ones)Uint8Arrayto base64 conversion)TestAttachmentinterface for custom metadatarecordArtifact(artifact, task)API for capturing artifacts during test executionAnnotations are now built on top of this system, though they maintain special internal handling for backward compatibility until the next major version. Everything should be fully backward compatible until then.
Note: Developing #8813 on top of this API provided a much better experience compared to the previous two attempts.
Please don't delete this checklist! Before submitting the PR, please make sure you do the following:
pnpm-lock.yamlunless you introduce a new test example.Tests
pnpm test:ci.Documentation
pnpm run docscommand.Changesets
feat:,fix:,perf:,docs:, orchore:.