Security boundaries

TraceDojo tests agent behavior in simulated environments. It is not a sandbox for hostile code, a security certification, or proof that a production service implements the same behavior. See LIMITATIONS.md.

Local execution and model calls

Adapters are trusted JavaScript modules. Importing an adapter executes code with the current process's filesystem, environment, and network access. Review code before running it. In-memory tool drafts isolate simulated state changes; they do not isolate the operating system or prevent external side effects.

Cancellation and time limits are cooperative. Blocking synchronous code and ignored cancellation cannot be forcibly stopped inside this process. The hosted dashboard accepts reports; it does not execute uploaded customer adapters.

Model-backed adapters send tasks, tool definitions, and conversation data to the configured provider. Use synthetic fixtures, minimize credentials in the process, and inspect what your adapter sends. The booking example disables automatic provider retries, records observed token usage, and stops on missing accounting. Token thresholds are not hard spending caps. Model observations come from trusted adapter code, not cryptographically signed provider evidence.

Pull-request execution and privileged comments

The generated test and comment workflows use separate jobs and separate trust boundaries. See the SDK setup guide for installation and configuration:

1. The test workflow checks out the PR head with credentials persistence disabled. It executes PR code with read-only repository/actions token permissions. The default starter needs no API key or production credentials. Do not add secrets to jobs executing untrusted PR code or run them on privileged, persistent self-hosted runners.
2. The candidate comparison uses saved evidence for the exact PR base SHA from the same workflow and repository. Missing or incomparable evidence cannot produce a passing regression verdict. Changing expectations can change what passes, so review workflow, tool, and assertion changes alongside agent code.
3. The workflow_run commenter has permission to write PR comments. It checks out only the default branch, with sparse checkout of its trusted script and credentials persistence disabled. It does not install dependencies or execute scripts from the PR or downloaded artifact.
4. The artifact is data, not trusted code. The commenter bounds the JSON file, rejects non-regular files, validates schema/version, commit labels, counts, and status consistency, and escapes displayed labels. Missing or invalid artifacts produce an incomplete result rather than a green comment.
5. GitHub API data identifies the associated open PR and its repositories. The commenter checks the current head/base and newer runs or reruns before writing. It updates only its own bot comment, not a human-authored marker.

The comment is a summary, not an attestation. Schema validation does not prove that a malicious PR ran honest tests. Protect the default branch, review CI changes, and require the test job through GitHub branch protection/rulesets if it should block merging. A failing job alone does not configure merge protection. The commenter does not grant approval or merge permission. Concurrent PR updates and API failures remain possible; use GitHub's current required-check status.

The shipped workflow retains evidence for 30 days, subject to repository settings. The separate opt-in live-model workflow uses seven days. Artifacts inherit GitHub repository access and can contain application data. GitHub Actions dependencies currently use version tags, not immutable commit pins; this remains a supply-chain trust dependency, not a claim of complete CI hardening.

Verification lives in workflow-comments.test.ts, workflow-ci.test.ts, and workflow-regression.test.ts.

Hosted accounts, ownership, and uploads

Email-code authentication uses Supabase Auth. Server handlers check the user and reject anonymous sessions; row-level policies constrain browser access by project ownership. Browser mutations require the configured application origin. The server-only database secret must never be exposed through browser configuration. It can bypass RLS, so server authorization and restricted database functions are part of the boundary—not just the presence of RLS.

Upload tokens are generated with cryptographic randomness, stored as hashes, bound to one project, and checked for expiry and revocation. They permit report uploads, not reading or deleting stored reports. Browser ownership sessions are required for deletion. Quotas, rate limits, bounded inputs, and private/no-store responses limit abuse and accidental caching; they are not a substitute for the deployment provider's network protections.

Apply all repository migrations: later migrations revoke broad inherited table and function grants. The initial schema alone is not the complete permission configuration. Cloud integration tests exercise ownership boundaries, token restrictions, and uploads against a fresh database. This document describes the implementation, not an independent penetration test.

Evidence, privacy, and retention

Reports undergo schema and consistency checks. Conflicting evidence under an existing identity is rejected, and identical imports are deduplicated. Owners can delete their hosted reports. Fingerprints identify test configurations; they do not establish who executed a test, prevent an owner deleting data, or provide signed, tamper-proof archival storage.

Pattern-based redaction removes recognized secrets and credential fields but is not comprehensive personal-data detection. Review reports before sharing, uploading, or attaching them to CI artifacts. Imported reports are externally supplied claims; validation does not independently establish that the recorded events occurred. Local servers bind to loopback and must not be exposed as hosted services.

There is no customer-configurable retention policy, legal hold, SSO, or compliance attestation. Keep required backups under your own policy. See privacy and storage for local persistence details.

Reporting a vulnerability

See Contact for the current support channel. Do not include exploit details, credentials, or customer data in an initial message. Request a private follow-up before providing a minimal synthetic reproduction. Do not test against other users or production data. There is no published bounty or guaranteed response-time commitment.