Project-related events in issues' timelines (V2 projects)

[omalkov]

Select Topic Area

Bug

Body

Hello,
I'm trying to retrieve an issue timeline for an issue added to a new (non-classic) GitHub project. I use the following query for this purpose:

query {
	repository(owner: "omalkov", name: "test.graphql") {
		id
		issue (number: 6) {
			id
			title
			timelineItems(first: 20) {
				edges {
					 node {
						 __typename
						... on AddedToProjectEvent {createdAt}
						... on RenamedTitleEvent {createdAt}
						... on IssueComment {createdAt}
						... on ClosedEvent {createdAt}
						... on AssignedEvent {createdAt}
						... on LabeledEvent {createdAt}
						... on ConvertedNoteToIssueEvent {createdAt}
						... on MovedColumnsInProjectEvent {createdAt}
					}
				}
			}
		}
	}
}

I added the issue to the project and changed the status of the issue several times. However, the query returns no events related to the project (AddedToProjectEvent, ConvertedNoteToIssueEvent, MovedColumnsInProjectEvent). I see other events, like the issue assignment or comments, but no project-related events. I expect that the issue timeline will contain all events, including the issue status changes in the project. I tried to add preview schemes, but it didn't help me. The query result looks like this:

{
	"data": {
		"repository": {
			"id": "R_kgDOHleUDg",
			"issue": {
				"id": "I_kwDOHleUDs5gRLWo",
				"title": "An Issue for the Statuses Test",
				"timelineItems": {
					"edges": [
						{
							"node": {
								"__typename": "AssignedEvent",
								"createdAt": "2023-03-08T11:30:08Z"
							}
						},
						{
							"node": {
								"__typename": "IssueComment",
								"createdAt": "2023-03-08T11:30:26Z"
							}
						}
					]
				}
			}
		}
	}
}

The same query works fine when I run it for an issue in a classic project.
Are the project-related events supported for the new (V2) projects? What I'm doing wrong?

[angjez]

There are also no timelineItems for DraftIssues which is a pity as the item might still be a draft until it goes into the review column.

For regular issues it just returns nothing anyway, I tried both the query in the main comment as well as the one below.

{
  user(login: "angjez") {
    projectV2(number: 1) {
      id
      title
      items(first: 20) {
        nodes {
          id
          content {
            ... on Issue {
              title
              timelineItems(itemTypes: [MOVED_COLUMNS_IN_PROJECT_EVENT, ADDED_TO_PROJECT_EVENT], first: 100) {
                nodes {
                  __typename
                  ... on MovedColumnsInProjectEvent {
                    id
                    createdAt
                  }
                  ... on AddedToProjectEvent {
                    id
                    createdAt
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

{
  "data": {
    "user": {
      "projectV2": {
        "id": "...",
        "title": "@angjez's untitled project",
        "items": {
          "nodes": [
            {
              "id": "...",
              "content": {
                "title": "Test TODO item",
                "timelineItems": {
                  "nodes": []
                }
              }
            },
            {
              "id": "...",
              "content": {}
            },
            {
              "id": "...",
              "content": {}
            },
            {
              "id": "...",
              "content": {}
            }
          ]
        }
      }
    }
  }
}

[ciemekk92]

The same happens in my case: I need the history of DraftIssues, yet there is none, and events AddedToProjectEvent and MovedColumnsInProjectEvent are not returned

[lukaszkamil]

I also found it missing in my case and this is basically a blocker for me right now :|

[danielzaucha]

Doesn't work for me either.

[krystianottobh]

I have the same problem, and since this data is crucial for analytics purposes, I'm surprised that it's not returned from an API.

[achodor]

I also have this problem, and it's blocking the progress of my project.

[ubiseb]

I am also looking for similar data. Is there any other ideas on how to get out cycle time of Issues in new projects (Ready - In progress - In review - In testing - Done)?

As we are a small team, I am now manually extracting the times from the Issue view for these two events;
Date+Time when issue "... moved this from 🔖 Ready to 🏗 In progress"
Date+Time when issue "... moved this from 🔨 Pending/In testing to ✅ Done"

And adding those to a script to calculate the time between.

[Pratheesha97]

I too need a means to monitor the duration of time spent on each issue status, starting from the issue opened date to the date it was resolved.

So this is a blocker for me right now as I need to access the specific dates and times when an issue is moved from one status to another which is missing in the response for new github projects. Do we have any other workarounds for this atm?

[jesuslinares]

Same issue. Related discussions:

• https://github.com/orgs/community/discussions/57326
• https://github.com/orgs/community/discussions/30979

[e0d]

There are several threads on this issue as @jesuslinares notes. Commenting here as it seems to have the most comments. It would be nice to get some update, say, an acknowledgment of the bug, some idea whether it will be fixed, how many quarters we should expect to wait.

[Galadirith]

I wanted to echo what @e0d shared. I really love using the new projects, and having these events actually captured is a relief so that in theory it is possible to understand the state of a Project at any given point in time. I've been trying for a few days trying different combinations of elements in the GraphQL API to try to see if there is some specific permutation that works. I would really appreciate even an acknowledgement of the issue so that I don't keep searching in hope for some magic combination that will unlock these objects and attributes.

I understand that part of this feature is supposed to currently be behind Schema preview, which doesn't work with the Explorer so you have to run separate from the Explorer. But part of this feature shouldn't require preview headers. And given that in general users can't any longer create Classic Projects

You can only create a new classic project board for an organization or user that already has at least one classic project board. You cannot create new classic projects for repositories. If you're unable to create a classic project board, create a project instead.

the only way for most users to make use of this documented feature is through new (non-classic) Project.

Really hacky workaround

I don't think this will be useful for large projects with many users and many associated repos. However it's the only solution that I have found so far that does part of what I need. I query the GraphQL API to get the last updated date of the "Status" of an issue

query {
  repository(owner: "Galadirith", name: "repo-name") {
    resourcePath
    issue(number: 239) {
      projectItems(last:100) {
        totalCount
        nodes {
          __typename
          updatedAt
          fieldValueByName(name: "Status") {
            ... on ProjectV2ItemFieldSingleSelectValue {
              name
            }
          }
        }
      }
      timelineItems (first:100) {
        updatedAt
      }
    }
  }
}

in one example I got back this

{
  "data": {
    "repository": {
      "issue": {
        "resourcePath": "/Galadirith/repo-name/issues/239",
        "projectItems": {
          "totalCount": 1,
          "nodes": [
            {
              "__typename": "ProjectV2Item",
              "updatedAt": "2023-08-19T10:23:45Z",
              "fieldValueByName": {
                "name": "Transfer"
              }
            }
          ]
        },
        "timelineItems": {
          "updatedAt": "2023-08-17T11:41:47Z"
        }
      }
    }
  }
}

showing the Status updating for the project, but not the timelineItems. This isn't great because this query is for a single project on a single issue on a single repo. I have not found a way to get something like "The last N ProjectV2Items from a project order by most recently updated", so for now I would have to run this query on all issue associated with a Project. Hopefully someone can suggest a better query.

From here I then filter the issues based on the projectItems updatedAt attribute. Now comes the really hacky bit. I grab some cookies from a GitHub session in my Browser and then run a Python request to download the issue HTML based on the resourcePath of all the filtered issues. Here is some barebones Python that I am using, again can definitely be done better and the headers might not work for you exactly like this

from bs4 import BeautifulSoup as BS
import requests
device_id = 'your-device-id'
user_session = 'your-user-session'
headers = {
    'authority': 'github.com',
    'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7',
    'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8',
    'cache-control': 'no-cache',
    'cookie': (
        'tz=Europe%2FLondon; '
        f'_device_id={device_id}; '
        f'user_session={user_session}; '
        'logged_in=yes; '
        'dotcom_user=Galadirith; '),
    'pragma': 'no-cache',
    'sec-ch-ua': '"Not/A)Brand";v="99", "Google Chrome";v="115", "Chromium";v="115"',
    'sec-ch-ua-mobile': '?0',
    'sec-ch-ua-platform': '"Windows"',
    'sec-fetch-dest': 'document',
    'sec-fetch-mode': 'navigate',
    'sec-fetch-site': 'same-origin',
    'sec-fetch-user': '?1',
    'upgrade-insecure-requests': '1',
    'user-agent': 'your-user-agent'
}
response = requests.get('https://github.com/Galadirith/repo-name/issues/239', headers=headers)

soup = BS(response.content, 'html.parser')
timeline_items = soup.find_all('div', class_='TimelineItem-body')
for item in timeline_items:
    if 'moved this from' in item.text:
        print(
            [s.get_text().strip() for s in item.find_all('strong')],
            [a.get_text().strip() for a in item.select('[href*="projects"]')],
            [t['datetime'] for t in item.select('.Link--secondary > relative-time')]
            )

That gives me back something like

['Explore', 'Transfer'] ['repo-name'] ['2022-12-05T12:10:31Z']
['Transfer', 'Orbit'] ['repo-name'] ['2023-08-15T07:42:53Z']
['Orbit', 'Transfer'] ['repo-name'] ['2023-08-15T07:43:47Z']
['Transfer', 'Orbit'] ['repo-name'] ['2023-08-19T10:23:30Z']
['Orbit', 'Transfer'] ['repo-name'] ['2023-08-19T10:23:45Z']

It's all really hacky, others might have tried something similar, but wanted to share as a terrible demo of some way to get some sort of really bad procedural logic into getting these invisible issue event.

References

• https://github.com/orgs/community/discussions/5859
• https://docs.github.com/en/graphql/overview/schema-previews#project-event-details-preview
• https://docs.github.com/en/graphql/overview/explorer
• https://docs.github.com/en/graphql/reference/objects#movedcolumnsinprojectevent
• https://docs.github.com/en/issues/organizing-your-work-with-project-boards/managing-project-boards/about-project-boards

[tino]

This unfortunately already doesn't work anymore, as they use some dynamic css class building....

[tino]

Any updates here? It's been two years and I still seem not to be able to just track when things were moved from column to column, which is very basic for getting out some cycle time...

[Fgerthoffert]

I created a separate discussion yesterday about adding to the timeline all events related to a card (and not just its status, which seems to be the subject of most related discussions).

I included a possible workaround on capturing and displaying change of status and anything else happening on a project card, but that's tedious to implement.

https://github.com/orgs/community/discussions/153239

[michalpavelka]

Can this be implemented, please? It doesn't sound like a whole lot of work to expose it through an API since it's already being displayed in the UI. I hope you are not expecting us to scrape the UI to get these details.

[Coroxx]

Hey, idk if it cans help we use a polling everyday at midnight to add a label on the concerned Issue. (For example, issue no. 242 was added today to the “Backlog” project. We therefore add a label named “Backlog” and delete it if it is no longer present in the project).

It allows us to get our timelines with Unlabeled and Labeled events 👍

But the real events would be appreciated

[andreaslech]

We ran into the same gap with the Projects (v2) API — there’s no direct way to get a full history of board status changes for issues.

As a workaround, I set up periodic polling to capture and store these transitions. This makes it possible to calculate metrics like Cycle Time and Time in State, which our Scrum Masters use for process improvement.

I built a small web app around it that visualizes issue movements (timelines, status changes, assignees). Overview here: Flow2C.com — or jump straight into the demo: app.flow2c.com.

I’d be interested in what other metrics or use cases you’d find valuable.

[rileybroughten]

You'll now see additional events in GraphQL including Status changes on an issue. Check out the changelog for more details!