PR Removed from GitHub Merge Queue Due to Status Check Timeout Despite Azure Pipeline Completing Successfully Within Timeout

[yuzhongqi]

Select Topic Area

Bug

Body

Observation:
The GitHub merge queue showed that a PR was removed from the queue with the message:
“github-merge-queue bot removed this pull request from the merge queue due to no response for status checks”
At that time, checking the Azure Pipeline history:
The pipeline triggered on the gh-readonly-XXX branch, created by the merge queue, had completed successfully.
The status in the GitHub merge queue still showed In Progress.
Later verification:
In the GitHub merge queue details, the [iOS] PR Build status was marked as Succeeded.
Summary:
The pipeline had completed successfully, but the merge queue removed the PR automatically after the 60-minute timeout because it did not receive the status update in time.
Possible Cause:
Status update delay / network or API issues:
The Azure Pipeline finished running on the gh-readonly-XXX branch, but the status update sent to the GitHub Checks API may have exceeded the merge queue’s 60-minute timeout.
Question:
Can we check GitHub provides logs for the merge queue showing how it retrieves the Azure Pipeline status.

[github-actions[bot]]

💬 Your Product Feedback Has Been Submitted 🎉

Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users.

Here's what you can expect moving forward ⏩

• Your input will be carefully reviewed and cataloged by members of our product teams. 
  • Due to the high volume of submissions, we may not always be able to provide individual responses.
  • Rest assured, your feedback will help chart our course for product improvements.
• Other users may engage with your post, sharing their own perspectives or experiences. 
• GitHub staff may reach out for further clarification or insight. 
  • We may 'Answer' your discussion if there is a current solution, workaround, or roadmap/changelog post related to the feedback.

Where to look to see what's shipping 👀

• Read the Changelog for real-time updates on the latest GitHub features, enhancements, and calls for feedback.
• Explore our Product Roadmap, which details upcoming major releases and initiatives.

What you can do in the meantime 💻

• Upvote and comment on other user feedback Discussions that resonate with you.
• Add more information at any point! Useful details include: use cases, relevant labels, desired outcomes, and any accompanying screenshots.

As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities.

Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐

[SafalNarsingh]

Step-by-step investigation: Why the PR was removed from the merge queue even though Azure Pipelines succeeded

1. Record the exact timeline (single source of truth)

What to do
Note these exact timestamps (UTC or your team timezone):

1. When the Azure Pipeline job finished (as shown in Azure Pipelines UI).
2. When Azure reported “status sent” (if present in the pipeline logs).
3. When GitHub recorded the check as completed (PR → Checks tab → expand the relevant check).
4. When the merge queue shows removed / timeout (merge queue message timestamp).
5. When the merge-queue branch gh-readonly-XXX was created and merged (if applicable).

Why

• The problem is a timing/ordering issue. Matching timestamps lets you see where a delay happened — whether the pipeline finished first and the status reached GitHub later, or whether GitHub never received the status at all.

Definitions

• Pipeline finished time: the time Azure reports the run completed.
• Checks tab completed time: when GitHub recorded the status check as finished.
• Merge queue removed time: when GitHub’s merge-queue bot gave up (60 min timeout).

2. Check GitHub’s PR “Checks” details (the place GitHub shows outcomes)

What to do

1. Go to the PR → Checks tab → find the Azure Pipeline / “[iOS] PR Build” entry → expand.
2. Look for timestamps: “Started at”, “Completed at”, and any job-level logs that say “reported status to GitHub” or show an event ID.

Why

• This shows what GitHub actually has recorded. If GitHub’s completed time is after Azure’s finished time, there was a delivery delay. If GitHub has no completion time, it never received the callback.

Definitions

• Checks API / Checks tab: GitHub’s UI and API surface that shows CI runs and their pass/fail status for a commit or PR.

3. Inspect GitHub webhook delivery history for the repository

What to do

1. Repo → Settings → Webhooks → open the webhook used by Azure DevOps (or the GitHub App) → check Recent Deliveries.
2. For each delivery, inspect:
3. Delivery timestamp
4. HTTP response code (200/201 vs 4xx/5xx)
5. Delivery latency
6. Any retry attempts and their timestamps

Why

• Azure Pipelines (or the Azure DevOps GitHub integration) typically notifies GitHub by sending an HTTP request. If those deliveries failed or were hugely delayed, GitHub wouldn’t see the check update in time.

Definitions

• Webhook delivery: the HTTP POST message sent by Azure/your CI to GitHub to communicate status.
• HTTP response code: tells you whether GitHub accepted the webhook (200) or returned an error (4xx, 5xx).

4. Inspect Azure DevOps service-hook / webhook logs

What to do:

1. In Azure DevOps: Project → Project Settings → Service hooks (or the pipeline’s post-job steps that call GitHub).
2. Look for the specific pipeline run → view its webhook delivery attempts, result codes, retries, and times.

Why

• This proves whether Azure attempted to notify GitHub on time and whether Azure received errors sending the status back.

Definitions

• Service hook: Azure DevOps’ mechanism to call external services (GitHub) on pipeline events.

5. Compare timestamps and identify the gap

What to do

1. Create a simple timeline:
2. Azure finished → [time delta] → webhook delivery attempts → [time delta] → GitHub recorded completion → [time delta] → merge-queue removed.
3. Highlight the longest gap.

Why

• The root cause is the longest gap. If the longest gap is between Azure finished and webhook sent, it’s an Azure issue. If it’s webhook sent to GitHub recorded, it’s network/delivery. If GitHub recorded completion but merge-queue removed later, there was an internal merge-queue mismatch.

Definition

• Time delta: the difference in minutes/seconds between events.

6. Check Azure Pipeline job logs for “reporting to GitHub” lines

What to do

• Open the pipeline run → expand job logs → search for “GitHub”, “Status”, “Checks API”, “report status”, or “create commit status”.
• Note any failure messages or long waits.

Why

• Many CI systems explicitly log when they POST status updates — if that step errored or retried, it will appear here.

Definition

• Job logs: the detailed console output for each CI step.

7. Verify the integration method and credentials

What to do

1. Confirm whether Azure is using:
2. GitHub App integration (preferred), or
3. Personal Access Token (PAT) / OAuth / Basic webhook
4. Verify the token/app is valid and not rate-limited or expired.

Why

• GitHub Apps have more reliable checks reporting (less likely to be throttled) than simple webhooks. A bad token or expired app can cause delayed or failed status updates.

Definitions

• GitHub App: OAuth-style integration with better authorizations and retries.
• PAT (Personal Access Token): token-based auth that can expire or have limited scopes.

8. Review GitHub repository audit logs / organization audit logs

What to do

1. If you have org-level access: Organization → Settings → Audit Log and search for merge-queue events and webhook events around those times.
2. Repo-level: Repo → Insights → Audit log (if available).

Why

• Audit logs can confirm higher-level events like “merge queue removed PR” and show who/what triggered removal. They won’t show webhook payloads but provide authoritative timestamps for GitHub-side events.

Definition

• Audit log: GitHub’s recording of important security and administrative events.

9. Try a repro / quick experiment

What to do

1. Create a small test repo and enqueue a PR through the merge queue.
2. Make a CI job that finishes quickly but artificially delays the webhook callback (or simulate a short network delay).
3. Observe whether merge queue times out at 60 minutes. Capture webhook delivery timings.

Why

• Reproductions confirm whether the merge queue strictly enforces the 60-minute window and whether similar delays cause removal.

Definition

• Repro: short for reproduction — repeating the situation to observe behavior in a controlled way.

10. If GitHub never recorded the completed check but Azure shows it finished on time

What to do

1. Inspect webhook deliveries and job logs. If those show success, contact GitHub Support with:
2. Repo name
3. PR URL
4. Exact timestamps you recorded
5. Evidence of webhook delivery (Azure logs + webhook delivery entries)
6. Request: “Please check merge-queue internal logs for delivery/processing of this check.”

Why

• GitHub internal logs (merge-queue internals) are not public. Support can inspect internal pipelines and tell you if GitHub had a processing delay or why the merge-queue bot removed the PR.

Definition

• GitHub Support: the official GitHub support team that can access internal server logs not available to users.

11. Short-term mitigations to avoid future unexpected removal

What to do & Why
Add a shorter-running “heartbeat” check: a tiny status check that completes fast and confirms the commit is alive — reduces risk of removal.
Why: small checks are less likely to be delayed.
Use GitHub App based status reporting instead of custom webhooks, if not already used.
Why: generally more robust and supports retries.
Make the pipeline report an initial “pending” status early and a “completed” later.
Why: merge queue sees a started check (so it knows CI is running) even if final completion is slightly delayed.
Instruct the merge-queue bot to re-enqueue automatically (if your automation supports it) on transient failures.
Why: transient network glitches should not require human re-queueing.

Definitions

• Heartbeat check: a tiny job that completes quickly to show CI is active.
• Pending status: an intermediate status showing work is in progress.

12. Long-term fixes and what to ask GitHub Support

What to do

1. When contacting support include:
2. Full timeline and artifacts (webhook delivery logs, Azure logs, PR URL).
3. Ask whether GitHub has any documented merge-queue timeouts other than 60 minutes and whether they can surface internal delivery logs for this event.
4. Ask if there are known transient incidents at the time (regional outages, API slowdowns).

Why

• Only GitHub can access internal queue processing logs and provide confirmation if the issue was on their side.

Definitions

• Internal delivery logs: logs inside GitHub infrastructure showing how events (webhooks/checks) were processed — NOT available publicly.

### TL;DR — plain English summary

1. GitHub’s merge queue will remove a PR if it doesn’t see required check results within 60 minutes.
2. GitHub does not publish internal merge-queue processing logs, so you cannot directly see “how the merge queue polled Azure.”
3. You can and should check:

• Azure pipeline finish time and job logs (look for “report status” lines).
• Azure service hook / webhook delivery times and response codes.
• GitHub repo webhook delivery entries and the PR Checks tab timestamps.
• GitHub audit logs for merge-queue removal event.

If webhook deliveries succeeded but GitHub still didn’t show the check in time, contact GitHub Support with the data — they can inspect internal logs.