Connectors

/

Azure DevOps

/

Compare Azure DevOps pipeline runs with Ghost Inspector suites
Report
Cross-Connector

Catch days when CI and browser tests disagree

For developers who want Azure DevOps pipeline runs checked against Ghost Inspector suite results, by asking their AI assistant.

Try PopdockAI free
Get a live demo
Compare Azure DevOps pipeline runs with Ghost Inspector suites
Compare the Contoso Web CD pipeline runs from the last week with the Checkout suite in Ghost Inspector.
3 of 12 pipeline runs disagree with the Checkout suite that followed them. - 11 Oct: pipeline passed, suite failed - 13 Oct: pipeline passed, suite failed - 15 Oct: pipeline failed, suite passed - Skipped: 1 run still in progress Each run is paired with the next suite result after it finished. Want the failing suite steps for those days?
Azure DevOps
Sample figures. Your results come from your own account.
azure-devops-get-pipeline-run
ghost-inspector-get-suite
ghost-inspector-get-suite-result

On this page

The problem

The problem

The prompt

Who it's for

How it works

Tools used

Connector

Azure DevOps

Azure DevOps
Run it on your own Azure DevOps data.

Connect Azure DevOps, pick your AI tool and paste the prompt.

Try PopdockAI free

The problem

A green pipeline can hide a red browser suite

CI pipeline status and end to end test results live in different tools. Days when the pipeline is green but the browser suite is red, or the reverse, go unnoticed because nobody lines the two up.

Without PopdockAI

  • Check pipeline runs in Azure DevOps
  • Check suite results in Ghost Inspector
  • Match them up by time by hand
  • Ship on a green pipeline with a red suite

With PopdockAI

  • Finished pipeline runs and suite results are read
  • Each run is paired with the next suite result
  • The pairing rule is stated
  • Pairs that disagree are listed, and both systems are only read

The prompt

Use this prompt

Copy it into any AI tool connected to PopdockAI. Swap the values in braces for your own.

Prompt
**Objective:** For the last `{lookback-days}` days (default 14), compare pipeline `{pipeline-name}` in Azure DevOps with suite `{suite-name}` in Ghost Inspector and report where one is green and the other is red.

**Step 1: Read the pipeline runs (azure-devops)**

Call `azure-devops-get-pipeline-run` with `name.contains` set to `{pipeline-name}`. If it matches more than one pipeline, list them and stop. Keep runs whose `created-date` is inside the window. Record `created-date`, `finished-date`, `result` and `state`.

**Step 2: Read the suite results (ghost-inspector)**

Call `ghost-inspector-get-suite` with `name.contains` set to `{suite-name}`. If more than one matches, list them and stop. Call `ghost-inspector-get-suite-result` with that `suite-id` and `date-created.from` set to the start of the window. Record `date-created`, `passing`, `count-failing` and `count-passing`.

**Step 3: Pair and compare**

Pair each pipeline run with the first suite result created after its `finished-date`, and before the next pipeline run finishes. Classify each pair: both green, both red, pipeline green with suite red, or pipeline red with suite green. Unpaired items go in their own list.

**Step 4: Report**

Counts per class, then the disagreement rows with both times and the failing count. Finish with one line on which direction the disagreements run, which says whether the pipeline gate is too loose or the suite is flaky.

**Rules & Guidelines:**

* Azure DevOps tools (prefix azure-devops-) come from the azure-devops server; Ghost Inspector tools (prefix ghost-inspector-) come from the ghost-inspector server. Both servers must be connected.
* Pairing is by time only; state that assumption.
* Do not call any write tool: `ghost-inspector-execute-test`, `ghost-inspector-execute-suite`, `ghost-inspector-execute-on-demand-test`, `ghost-inspector-update-test`, `ghost-inspector-update-suite`, `ghost-inspector-add-test-result-comment`, `ghost-inspector-add-suite-result-comment`, `ghost-inspector-accept-screenshot`, or any create, duplicate, cancel or update tool. This prompt only reads.

**Validation:**

* If either side has no rows in the window, report that and stop.
* A pipeline run with `state` not completed is still running; leave it out and say so.

Who it's for

One answer, three teams that use it

QA engineers

See where tests and builds disagree

Pairs that disagree are listed with times, so you know which runs to investigate.

Compare the Fabrikam API CI runs this week with the Smoke suite in Ghost Inspector.

Developers

Find the change that broke the browser tests

Know which pipeline run came before a failing suite, so you can look at the right change.

When did the Northwind Portal pipeline pass but the Login suite fail in Ghost Inspector?

Release managers

Check quality before you ship

Confirm both CI and end to end results are green for recent runs before a release.

List days in the last two weeks where the Contoso Web CD pipeline and Ghost Inspector Checkout suite disagreed.

How it works

What PopdockAI does when you ask

You name a pipeline and a suite, and your AI assistant pairs their results over time. It stops if a name is ambiguous and changes nothing.

01

Ask

You name the pipeline and the Ghost Inspector suite.

Your AI tool

02

Read

It reads recent finished pipeline runs and suite results.

azure-devops-get-pipeline-run
ghost-inspector-get-suite
ghost-inspector-get-suite-result

03

Pair

It matches each run with the next suite result and states the rule used.

No items found.

04

Report

You get the pairs that disagree, with times and results.

Your AI tool

What PopdockAI provides

The tools behind the answer

Ready made tools for Azure DevOps and one secure connection to your AI tool. No code, no exports.

get-data

azure-devops-get-pipeline-run

Returns pipeline runs records from the Azure DevOps connector. Use it to read and filter pipeline runs across your Azure DevOps projects.
get-data

ghost-inspector-get-suite

Returns the test suites defined in Ghost Inspector, with the folder and organization they sit in and the browser, region and start URL their tests run with. Filter by organization, folder, name, browser or region, or fetch a single suite by ID.
get-data

ghost-inspector-get-suite-result

Returns Ghost Inspector suite runs, with the number of passing and failing tests in each, the run timing and the suite they belong to. Filter by suite, date range, pass state, or a minimum count of failing tests.
Read only.Nothing in Azure DevOps Changes

This report is built from Get data tools only. PopdockAI applies the row, field and parameter rules your admin sets, and every query is written to the audit trail.

How Popdock AI secures your data

The outcome

No red suite hiding behind a green pipeline

Try PopdockAI free
Get a live demo

Related use cases

More ways to put this data to work

All Ramp use cases

One prompt template with infinite possibilities.

‍

Your AI assistant adapts its approach, tone, and capabilities to match each unique context while you maintain a single, manageable workflow.

Join forward-thinking companies that are transforming their operations with intelligent, secure AI automation.

Try PopdockAI Free
See Documentation