Runs this week
446
↑ 12.8% vs last week
Good morning, Maya
Here’s how your tests did this week, Oct 31 – Nov 6.
3/4
Almost set up — connect CI to run on every change
Your tests already run locally. One step left to run them on every push and pull request.
Install the CLIRecord your first testAdd a safety policy- 4Connect CI
Pass rate
96.2%
↑ 0.6 pts vs last week · 429 of 446 passed
Requests stopped
17
by the safety door — none reached a real service
Locators healed
23
average match 91% · 3 to review
Run activity
Runs per day by result, last 7 days
- Passed
- Policy blocked
- Failed
Safety door this week
Every request your tests tried to make
1,962
allowed
17
stopped
412
keys swapped
Most often stopped
- rule block-destructive-refunds9
- rule block-admin-mutations5
- host on the blocked list3
Needs attention
Latest signed receipt
run_40549cd06baf · Nov 6, 08:40
- Merkle root
- e2114ac9fab789cb…a8f6aece
- Signature
- HMAC-SHA256 · 3c05ca9c…0a24
- Ledger
- 5 blocks · 1 request blocked · 2 stand-in keys
Runs
Every test run — on laptops, in CI and on cloud runners. Each one leaves a signed receipt.
Checkout payment
Failed at step 17Run #1039 · Nov 6, 09:31 · triggered by CI pipeline on pull request #128
What happened, in plain words
The test logged in, filled the cart and applied the SAVE20 coupon. When it clicked Pay now, your server answered with an error (500), so the order confirmation never appeared. Along the way the safety door stopped a refund request and an unknown website, exactly as your policy says. One renamed field was found again automatically.
Tests
Your suites and the specs inside them — and how steady each one has been.
Suites · 5 Sample suite groups
Analytics
Are tests getting steadier, faster and safer? Every number below follows the filters.
Pass rate
96.8%
↑ 1.1 pts vs previous 30 days
Flaky rate
1.9%
↓ 0.4 pts fewer reruns
Median run
41s
↓ 6% faster than before
Requests stopped
61
0 reached a real service
Pass rate
Share of runs that passed, last 30 days
How long runs take
Number of runs by duration, last 30 days
Median 41s · runs that restore a checkpoint skip their setup and land in the shortest buckets.
What the safety door stopped
Stopped requests by reason, last 30 days
rule block-destructive-refunds38
rule block-admin-mutations21
blocked host13
not on the allowed list8
secret blockedPaths4
How broken locators were found again
Healed steps by matching method, last 30 days
Same test idexact_testid38
Accessible namearia_fuzzy29
Visible textsemantic_text17
Page structurestructural_xpath8
Similarity scoreml_heuristic4
Heals below the 80% confidence threshold are never applied — they fail the step and appear in review.
Record a test
Click through your app like a customer would. AirlockQA writes down each step and turns it into a Playwright test.
- ✓Record
- 2Review steps
- 3Save as a test
New recording
Prefer the terminal?npx airlock explore http://localhost:3000 --name checkout-flow
Self-healing
When a button is renamed or moved, AirlockQA finds it again so the run can continue — then asks a person before the fix becomes part of your test.
Healed this week
23
runs kept going instead of failing
Average match
91%
confidence across applied heals
Waiting for review
3
accept to update the spec
Couldn’t heal
2
best match below the threshold
Review queue
Checkout › applies the SAVE20 coupon
Step 6 · type into a field · seen in run #1039 · React
Was looking for
#coupon
Found instead
getByTestId('coupon-input')
Match94%
before <input id="coupon" name="coupon">
after <input data-testid="coupon-input" name="promo">
Account › updates the display name
Step 3 · click · seen in run #1038 · React
Was looking for
button.btn-save
Found instead
getByRole('button', { name: 'Save changes' })
Match88%
before <button class="btn-save">Save</button>
after <button class="button primary">Save changes</button>
Checkout › guest checkout
Step 9 · click · seen in run #1035 · Vue
Was looking for
text=Continue as guest
Found instead
getByText('Checkout as guest')
Match81%
before <a href="/guest">Continue as guest</a>
after <button>Checkout as guest</button>
Checkpoints
Save points for your app. Capture a moment — signed in, cart filled — then start new tests from it, side by side, without repeating the setup.
Saved in every checkpoint
Cookies, local storage, session storage, the page URL and a DOM snapshot with its hash.
Not saved
Your server’s data, background jobs, or process and VM memory. Branches share the same backend.
Saved checkpoints
.airlock/checkpointsBranch from cart_ready
Each branch starts in a fresh browser context restored from the checkpoint, then tries one variation.
$ npx airlock fork cart_ready --workers 4
Latest branch results
4 branches · 4 at a time · run 1cart_ready
setup skipped in every branch · saved 24.8s of repeated setup
- Apply coupon SAVE20Passed1.4sReached /checkout/review with a $50.00 totalassertions 3/3 · ledger 4be1c9…
- Expired couponHandled1.1sShowed “This coupon has expired” and kept the full priceassertions 3/3 · ledger 91c07a…
- Guest checkoutFailed1.6sExpected /checkout/guest but landed on /loginassertions 2/3 · ledger 7d2a18…
- Pay with saved cardPassed1.3sCard ending 4242 preselected, stand-in key swapped at the doorassertions 3/3 · ledger e05f3b…
Request policy
Your tests’ safety door. Decide what the test browser may reach, and give it stand-in keys so real ones never enter the browser.
When no rule matches
What should happen to a request headed somewhere you haven’t listed?
"defaultAction": "BLOCK"Hosts
Allowed
- localhost
- 127.0.0.1
- api.stripe.com
- staging.shop.test
Always blocked
- pastebin.com
- webhook.site
- transfer.sh
The safety door covers requests made by the test browser. Calls your own servers make aren’t routed through it.
Evidence
Every run writes a step-by-step log, links each entry to the one before it, and signs the result. Change a single entry and the receipt stops checking out.
Latest receipt
run_40549cd06baf
Nov 6, 2026 · 08:40:27 UTC · local run · 393 ms
- Merkle root
- e2114ac9fab789cbffd7cebba2fab189 58a23ec41bbdc0af7a0f3f1ba8f6aece
- Signature · HMAC-SHA256
- 3c05ca9c533fb6903624f8188ec698b9 13c3bfad569c8c8562b74efcc2ef0a24
- Signing key
- airlock-signer-key-id-01
5
ledger blocks
2
stand-in keys configured
1
request blocked
0
requests allowed out
Not measured
real secrets in sandbox
The ledger, link by link
Each block stores the hash of the block before it. Try changing one to see why tampering shows.
- #0linkedEXEC_STARTRun began · checkout-flowprev 00000000hash 4f1c2a9e
- #1linkedSECRET_SWAPSTRIPE_KEY stand-in replacedprev 4f1c2a9ehash 9b02d7c1
- #2linkedEGRESS_BLOCKEDPOST /v1/refunds stoppedprev 9b02d7c1hash c7e95b30
- #3linkedASSERTION_EVALTotal shows $50.00 · passedprev c7e95b30hash 21ad8f64
- #4linkedEXEC_COMPLETERun finished in 393 msprev 21ad8f64hash e8c6c259
Every block points at the hash of the one before it, and the Merkle root of all blocks is signed. Nothing has changed since signing.
Verify a bundle
Checks the signature with your key and recomputes the Merkle root — the same check as airlock audit.
Drop an attestation.json here
or
latest-attestation.json
Where this evidence can help an audit
AirlockQA produces evidence. It doesn’t assess or certify controls — your auditor does.
- CC6.1Logical access stand-in keys, host allow-listsNot assessed
- CC6.6Boundary protection blocked and allowed request recordsNot assessed
- CC7.1Change verification signed results per commitNot assessed
- CC8.1Non-repudiation hash chain and signatureNot assessed
Example identifiers are not signed evidence. Cryptographic proof is verified against the SHA-256 Merkle root.
Time warp
PREVIEWSee what your app shows days from now — trials ending, coupons expiring, renewal reminders — without waiting for the calendar.
What moves: the clock that JavaScript’s
Date reads inside the test page, including pages it opens next. What doesn’t: browser timers, your servers, databases, cron jobs and certificates keep real time — so server-side expiry needs its own test.Move the page clock
Page time now
Nov 6
2026 · 10:42
After the warp
Nov 20
2026 · 10:42
Jump ahead by
$ npx airlock warp "+14 days"
What the test page shows
staging.shop.test/account · trial started Oct 30
TimeWarpResult
scope browser-page
previousTimeIso 2026-11-06T10:42:00.000Z
targetTimeIso 2026-11-20T10:42:00.000Z
deltaMs 1209600000
affectedPageAPIs Date, Date.now()
ledger TIMEWARP_ADVANCED
Good uses — and one to avoid for now
Trial or plan banners
The page computes “days left” from Date — a perfect fit.
Coupon and sale end dates
Check the countdown and the “expired” message shown in the browser.
Real subscription expiry
Your server decides when a plan ends, and its clock doesn’t move. Test that on the server side.
Fixtures
PREVIEWSave an outside service’s answer once, then replay it in every run — so a test doesn’t fail just because that service had a bad day.
Fixtures answer requests made inside the test browser. Your servers’ own calls aren’t captured, and a request with no saved answer currently goes out to the real network — every miss is listed below so nothing slips by unnoticed.
Mode for the next run
Block requests with no saved answerStrict mode is on the roadmap
Saved answers
5
1 synthetic
Served this week
1,140
requests answered locally
Misses
2
went to the real network
Chaos report
PREVIEWFind buttons that could charge, submit or send twice — before a customer double-clicks or a network hiccup retries.
Today the check watches one real click and looks for an idempotency key on the request it sends. Duplicate, dropped-connection and 502 scenarios are described, not executed yet — each row says which.
What to check
npx airlock chaos http://localhost:3000Risk indicated
Checkout can be sent twice
POST /api/checkout went out without an Idempotency-Key header. If a customer double-clicks or the browser retries, your server can’t tell the two apart.
Suggested fix
Send a unique Idempotency-Key with each checkout request, and have the server return the first result for any repeat.
This is a warning sign, not proof of a double charge — your server may already guard against repeats in another way.
request headers seen
content-type application/json
authorization Bearer ph_sec_stripe… (stand-in)
traceparent 00-4bf92f35…-01
idempotency-key — missing
Bug capsules
PREVIEWWhen something breaks in a test environment, a capsule keeps the last moments before the error. AirlockQA turns it into a Playwright test that reproduces the bug.
Prototype — keep real customer data out. Emails, card numbers, tokens and obvious secrets are scrubbed on a best-effort basis, but capsules aren’t encrypted and storage can still hold sensitive values. Use staging or preview data only.
Captured · 3
How a capsule is made
- Your test app keeps a small rolling log of recent clicks, inputs, requests and console messages.
- When an error isn’t caught, it saves that log as a capsule with a checksum.
- Teleport replays the steps in a fresh browser and writes a repro test.
Checkout crashed after applying a coupon twice
TypeError: Cannot read properties of undefined (reading 'total')
- Environment
- staging · app 2.14.0
- Page
- /checkout
- Browser
- Chromium · 1280×800
- Checksum
- sha256:5be0…a91c
Scrubbed before saving
[REDACTED_EMAIL] × 2[REDACTED_PAYMENT_CARD] × 1[REDACTED_JWT_TOKEN] × 1Review: 3 storage keys, 2 cookies kept
The last moments before the error
- −4.8snavigation/checkout
- −4.1sdom_inputinput[name=email] → [REDACTED_EMAIL]
- −3.6sdom_inputinput[name=coupon] → "SAVE20"
- −3.2sdom_clickbutton “Apply coupon”
- −2.9snetwork_reqPOST /api/cart/coupon → 200
- −1.4sdom_clickbutton “Apply coupon” (again)
- −1.1sconsolewarn: coupon already applied
- 0.0suncaught_errorTypeError: Cannot read properties of undefined (reading 'total') · checkout.js:212
Connect your pipeline
Run AirlockQA on every push and pull request, and get the result — with its signed receipt — right on the pull request.
1 · Where should tests run?
2 · Install
$bun add --dev airlockqa $ bunx airlock test --cloud3 · Store the key as a CI secret
Secret nameAIRLOCK_API_KEYGitHub → Settings → Secrets → Actions
- The CLI sends the key only as an HTTPS Bearer token — never in the run body or a command argument.
- Don’t expose the secret to workflows triggered by pull requests from forks.
- Real app secrets stay in your runner; only stand-in values travel to the cloud.
5 · See results on your pull requests
airlockqa bot commented on #131All 3 suites passed · 72 of 72 steps · 38s
2 requests stopped by policy · 1 locator healed (needs review) · receipt e2114ac9… signed
Cloud usage today
acme-web64 / [LIMIT] runsWhole workspace91 / [LIMIT] runsEach parallel worker and each retry counts as a run. Limits reset at midnight UTC; the CLI warns when 20% or less remains.
EXECUTION FLOW · THREE.JS ARCHITECTURE
From commit to verified attestation
Three.js CI to attestation flow
POST
/v1/runs
· Cloud run submission endpoint
Settings
Workspace details, people, run defaults and how evidence is signed. Everything here maps to airlock.config.json.
General
Projects
- acme-web3 suites
- acme-admin1 suite
- notes-app1 suite
Members
Admins manage keys and policy · Editors record and run tests · Viewers read results and receipts
| Person | Role | Last active |
|---|---|---|
| MCMaya Chenmaya@acme.test | Admin | Now |
| JLJordan Leejordan@acme.test | Editor | Today, 09:54 |
| PNPriya Nairpriya@acme.test | Viewer | Nov 5 |
| CI pipelineservice · uses key air_live_7Hk2… | Runner | 2 min ago |
Run defaults
Used by every run unless a suite overrides them
- Run browsers without a windowFaster in CI. Turn off to watch a run locally.
browser.headless - Safety door onEvery browser request passes the request policy.
proxy.enabled - Add accessible-name fallbacks to compiled testsGives each step a second way to find its element.
compiler.emitAriaFallbacks - Self-heal broken locatorsApplies a fix at 80% match or higher, then asks for review.
healer.autoHeal - Save every heal to the datasetPowers the healing log and airlock dataset.
healer.saveMutations
airlock.config.json
Real secret values are never shown or uploaded — they stay in your runner.
{
"projectName": "Acme SaaS QA",
"baseUrl": "https://staging.shop.test",
"browser": { "headless": true, "viewport": { "width": 1280, "height": 800 } },
"proxy": { "enabled": true, "port": 8899 },
"secrets": {
"STRIPE_KEY": { "real": "•••• kept in your runner", "phantom": "ph_sec_stripe_test_placeholder", … }
},
"policy": { "defaultAction": "BLOCK", … },
"compiler": { "defaultTimeoutMs": 10000, "emitAriaFallbacks": true },
"healer": { "autoHeal": true, "confidenceThreshold": 0.8, "saveMutations": true },
"forker": { "maxParallelForks": 5 },
"compliance": { "outputDir": "./.airlock/compliance", "generateHtmlReport": true }
}Evidence signing
How each run’s receipt is sealed
- Key ID
- airlock-signer-key-id-01
- Receipts folder
- ./.airlock/compliance
- Last rotated
- Oct 30, 2026 by Maya Chen
Never use the sample key from the docs outside a demo.
Cloud plan Early access
[YOUR PRICE] / month · daily runs: [YOUR LIMIT] per repository
Running locally is always free and doesn’t count toward cloud usage.