Six layers of tests
Manifest and structure
Confirm the bundle has valid instructions, typed inputs and outputs, and declarations for required tools.
Sandbox execution
Run scripts in isolation and review whether observed access matches the manifest.
Golden-set tests
Score known inputs against expected outputs and cover each declared failure mode.
Adversarial cases
Check how the skill behaves when it receives misleading inputs or encounters edge cases.
Regression checks
Compare the candidate with its predecessor to see where results improved or declined.
Compatibility
Record which models and agent tools were exercised by the test suite.
Continuous re-testing
A result belongs to the model, tools, and cases that produced it. Re-test when they change.
| Change | Review |
|---|---|
| Model change | Check the release with the new model |
| Tool or connector change | Re-run affected cases |
| New golden cases | Compare coverage and expectations |
| Scheduled review | Set a cadence that fits the workspace |
Test on your own work
Add representative cases
Use examples drawn from the work your team already reviews.
Set acceptance thresholds
Decide the minimum score and regression tolerance for each trust tier.
Compare before rollout
Run an evaluation against expected outputs before granting production access.
What auditors receive
| Record | What it shows |
|---|---|
| Suite version | Cases and approver |
| Results by case | Inputs, expected and actual outputs, and score |
| Regression comparison | Changes from the prior release |
| Sandbox trace | Observed access |
| Release decision | Applied threshold and approval |