Remove x-gha binary cache provider
#1662
Conversation
IOW they changed their mind since #836 (comment) / #836 (comment). |
It's less 'we changed our mind' and more 'the service we were told was OK got totally sunset and the replacement is entirely different' |
src/vcpkg/binarycaching.cpp
Outdated
| handle_readwrite(state->gha_read, state->gha_write, segments, 1); | ||
|
|
||
| state->binary_cache_providers.insert("gha"); | ||
| return add_error(msg::format(msgUnknownBinaryProviderType), segments[0].first); |
There was a problem hiding this comment.
Add an error message that summarizes reasons for removal and alternatives.
|
Is there a reason that this won't work? (See microsoft/vcpkg#45116 (comment))
|
The owners of that API have indicated that there is no stable API we could use for this use case and that the only "paved path" they won't regularly break is (The new API also would require adding a huge number of dependencies to vcpkg e.g. protocol buffers even if we wanted to attempt that) |
|
OK, thanks for the additional info. I hadn't realized protobuf would be required The nuget tutorial for package caching is lacking some information, I filed a doc bug on improving that. Basically, the recipe given was one that would only work for linux. |
I don't see anything restricting to Linux in https://learn.microsoft.com/vcpkg/consume/binary-caching-nuget ; can you clarify? |
|
Look here instead: https://learn.microsoft.com/en-us/vcpkg/consume/binary-caching-github-packages I wrote up a gist that explains how I got everything working: |
@LegalizeAdulthood Did you try the Windows/Linux toggle at the top?
|
The unplanned dropping of the granular vcpkg binary cache indeed fell into the cracks between Microsoft's various departments. The old method is now official dropped, without replacement either on the vcpkg side or the GitHub cache provider side. Without a granular cache, vcpkg is impractical for builds larger than a small dependency tree in CI, for performance reasons. A granular cache is critical for CI use. Building dependencies is not a goal of this CI, so a more desirable option would be pre-built binary downloads. This would also allow keeping job timeouts low, which is important for quick iteration in GHA when a flaky job requiring a manual retry needs all other jobs to finish first. (GHA often disregards step timeouts, which is another contributing factor here.) Windows remains tested extensively with MSYS2, curl-for-win, and via AppVeyor CI with MSVC + OpenSSL, and also in GHA via scaled back vcpkg jobs that perform well without caching. What's lost is the recently added Android OpenSSL build tests. We may consider building/cachine important dependencies manually as in GHA/linux-http3, and/or try integrating MSVC jobs with MSYS2 UCRT DLLs. Ref: microsoft/vcpkg-tool#1662 Ref: microsoft/vcpkg#45073 Follow-up to cd0ec47 #17089 Follow-up to e3912f0 #17086 Follow-up to 15fb1dc #17069 Closes #17200
BillyONeal
left a comment
BillyONeal
left a comment
There was a problem hiding this comment.
I'm sad that this is broken with no practical replacement but it is what we've got :(
Oh, you mean the button that's way far away from the actual code such that it is off screen while I'm reading the code? No, never even noticed it. |
I'm trying to get as much as possible of the multiple steps needed to make NuGet packages work across {macos,ubuntu,windows}-latest into a composite action. A rough first draft is here: The part that is a pain is setting up the access token(s) more than the steps, though. It also makes me think that it's going to be more difficult for other people to run your workflow with caching in their fork. |
|
On Linux, you need mono to execute nuget.exe， but mono is dead. |
It's working OK for me in the github runners; can you elaborate? |
|
See: NuGet/Home#12513 |
|
er... I think I follow what's going on in that issue, but if we can't use |
CI has been failing for the last couple of days with caching related failures. See for example : https://github.com/TileDB-Inc/TileDB/actions/runs/15327048347/job/43143363036?pr=5522 - Investigation showed that this is due to: "Recent [changes in GitHub Actions Cache's API](actions/cache#1510) have rendered the x-gha binary caching provider useless." From: microsoft/vcpkg-tool#1662 . In this PR ways to replace this cache with alternatives (e.g. NuGet) are suggested. - Microsoft put a warning in the vcpkg page: https://learn.microsoft.com/is-is/vcpkg/consume/binary-caching-github-actions-cache that x-gha cache is no longer supported. - Github-actions have warned about messing up with `ACTIONS_` env variables here: https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/#decommissioned-cache-service-brownouts For now we'll disable x-gha. We can open a follow up ticket to investigate/implement alternatives if needed. closes CORE-231 --- TYPE: NO_HISTORY DESC: Disable x-gha binary cache for vcpkg
|
I maintain https://github.com/johnwason/vcpkg-action which has an option to use this feature and found the tests were failing. My old strategy runs the vcpkg cli twice. The first run to capture a list of the packages to build and uses this as a cache key. This cache key stores the local binary cache files from vcpkg. The second run can then use these files, or update a new cache if the cache key doesn't exist. It isn't a perfect solution but it does appear to still work. |
It sounds like we could benefit from having a "JSON file API" for vcpkg, similar to what we have for CMake, whereby external tools could query vcpkg about things like list of files to be cached as sources or binaries for a port. |
|
It appears you can use the actions/cache as a node API to load and store caches. See this action for an example: https://github.com/hendrikmuhs/ccache-action |
Yes, but it's up to you to identify which sets of files are to be cached together with the low-level cache action. With vcpkg what you want is to cache all the files related to a dependency and it's associated hash (e.g. triplet plus maybe a hash of the port directory contents). This is why I was thinking it would be beneficial to have a "JSON file API" to vcpkg. vcpkg knows what files go with what dependency and if these file lists were available externally to a caching tool, then CI caching mechanisms could be decoupled from vcpkg entirely. |
|
An option to save a text file with one vcpkg cache filename per line alongside --dry-run would probably be enough for my use case. |
|
You can use something like this: lukka/run-cmake#152 (comment) |
10 participants
Recent changes in GitHub Actions Cache's API have rendered the
x-ghabinary caching provider useless.In a chat with GitHub Actions engineers, the following points were made:
Given these reasons, we have decided to remove support for GitHub Actions Cache as a binary caching provider. This may change if a public API ever becomes available.
Some alternatives to migrate from
x-gha:actions/cachein your workflow to cache theinstalleddirectory, this is not integrated with vcpkg but is the method endorsed by the GitHub Actions team. The downside of this method is that the whole installed tree gets cached as a single artifact, so single port updates invalidate all packages.