/
/
Name the flaky Ghost Inspector tests and what causes them
For QA teams who want to tell flaky tests from broken ones using real run history, by asking their AI assistant.
On this page
Connector
Connect Ghost Inspector, pick your AI tool and paste the prompt.
The problem
Flaky tests erode trust in the whole suite
Flaky Ghost Inspector tests are hard to see because each failure looks like a one off. Telling them apart from broken tests, or from failures in one browser, takes a run history nobody has time to assemble.
Without PopdockAI
- Rerun a failing test and hope it passes
- Collect pass and fail history by hand
- Compare browsers and regions one by one
- Learn to ignore red results
With PopdockAI
- Tests that both passed and failed are kept, always failing ones ignored
- A flakiness rate is worked out for each
- Retry, screenshot, browser and region settings are checked
- Mitigations are recommended, not made
The prompt
Use this prompt
Copy it into any AI tool connected to PopdockAI. Swap the values in braces for your own.
**Objective:** Identify flaky Ghost Inspector tests over the last `{lookback-days}` days (default 14), counting only tests with at least `{min-runs}` runs (default 5).
A flaky test is one that both passed and failed in the window without its configuration changing. A test that fails every time is broken, not flaky, and belongs in a different report.
**Step 1: Collect both outcomes**
Call `ghost-inspector-get-test-result` with `passing` true and `date-created.from` at the start of the window, then again with `passing` false and the same date filter. Set `max-records` high enough to cover the window in both calls.
**Step 2: Find the mixed ones**
Group by `test-name` across both sets. Keep only tests that have at least one pass and at least one failure and at least `{min-runs}` runs in total. Compute a flakiness rate as failures over runs. Ignore tests at 100 percent failure.
**Step 3: Check whether the configuration explains it**
For each candidate call `ghost-inspector-get-test` with `name.contains` set to that test name and read `auto-retry`, `screenshot-compare-enabled`, `browser` and `region`.
* if failures cluster in one `browser` or `region` it is an environment difference, not flakiness
* `screenshot-compare-enabled` true and `screenshot-compare-passing` false on the failing results means the flakiness is visual diff noise, not a functional failure
* `auto-retry` false on a test that is only intermittently failing is the cheapest available mitigation
**Step 4: Look at where they break**
For the three flakiest, take the most recent failing result `id` and call `ghost-inspector-get-test-step` with that `test-result-id` and `passing` false. A flaky test that always breaks on the same step is a timing problem on that step; one that breaks in different places is an environment problem.
**Step 5: Report**
A table of: test name, suite, runs, failures, flakiness rate, auto retry, screenshot compare, browsers affected. Then for the top three, the step they break on and a recommended mitigation.
**Rules and guidelines:**
* Read only. Do NOT call `ghost-inspector-execute-test`, `ghost-inspector-execute-suite`, `ghost-inspector-update-test` or any create, duplicate or comment tool. Recommend the setting change, do not make it.
* Do not call a test flaky on two runs. Respect the `{min-runs}` floor and say how many candidates you dropped for want of data.
**Validation:**
* If no test has both a pass and a failure in the window, say the suite set is stable over that period and stop.
* If a test name matches more than one test in step 3, say so and treat the configuration check as inconclusive for it.
Who it's for
One answer, three teams that use it
Test automation engineers
Know which tests are flaky
Tests ranked by flakiness rate, with the setting or step behind it, show where to start.
Which Ghost Inspector tests flipped between pass and fail in the last 30 days, with at least 10 runs?
QA leads
Tell timing problems from environment ones
Where the worst tests break shows whether to add waits or fix an environment.
For the three flakiest Ghost Inspector tests this month, are the failures timing or environment related?
Developers
Stop ignoring red results
Once flaky tests are named and fixed, a red result means something again.
List flaky Ghost Inspector tests in the Checkout suite and what you would change for each.
How it works
What PopdockAI does when you ask
You give a period and a minimum number of runs, and your AI assistant names the flaky tests with evidence. Tests below the run floor are dropped and counted.
01
Ask
You give a lookback period and a minimum number of runs.
02
Collect
It gathers passing and failing results and keeps tests that did both.
03
Check
It reviews retry, screenshot, browser and region settings and where the worst ones break.
04
Report
You get flakiness rates and recommended changes, and nothing is changed.
What PopdockAI provides
The tools behind the answer
Ready made tools for Ghost Inspector and one secure connection to your AI tool. No code, no exports.
ghost-inspector-get-test-result
ghost-inspector-get-test
ghost-inspector-get-test-step
This report is built from Get data tools only. PopdockAI applies the row, field and parameter rules your admin sets, and every query is written to the audit trail.
The outcome
A suite you can trust again
Related use cases
More ways to put this data to work
One prompt template with infinite possibilities.
Your AI assistant adapts its approach, tone, and capabilities to match each unique context while you maintain a single, manageable workflow.
Join forward-thinking companies that are transforming their operations with intelligent, secure AI automation.

.webp)
