Add universal2 builds of Python to actions/setup-python

[bwoodsend]

Tool name

universal2 builds of Python for macOS

Tool license

Python software foundation

Add or update?

• Add
• Update

Desired version

latest (3.9.6)

Approximate size

40MB in .pkg form. 150MB installed

Brief description of tool

As you're probably aware, Apple are transitioning from x86_64 to arm64. Whilst x86_64 machines are still around, Apple are encouraging software authors to ship universal2 applications which are executables for both x86_64 and arm64 glued together into one file and the OS is able to select the right half of the executable to run at runtime. The latest versions of Python on https://python.org/downloads now have a macOS 64-bit universal2 installer download option.

Given that the CI/CD servers are all x86_64, you're probably wondering why any of the above matters. Well... Python package authors are now also required to build their C extensions for macOS universal2 too. On the plus side, Python does this all automatically if the Python installation itself is compiled universal2. On the down side, there is no other way to build universal2 Python packages.

So long story short, without trying to do some wget https://www.python.org/ftp/python/3.9.6/python-3.9.6-macos11.pkg; sudo installer -pkg python-3.9.6-macos11.pkg -target / scariness to install Python directly from python.org, it is impossible to build Python packages from macOS arm64 on Github Actions.

One thing to note, building universal2 Pythons does not require access to arm64 machines - rather they can be cross compiled provided you have XCode >= 12.2 installed.

URL for tool's homepage

No response

Provide a basic test case to validate the tool's functionality.

test `lipo -archs $(which python)` = "x86_64 arm64"

Virtual environments affected

• Ubuntu 16.04
• Ubuntu 18.04
• Ubuntu 20.04
• macOS 10.15
• macOS 11
• Windows Server 2016
• Windows Server 2019
• Windows Server 2022

Can this tool be installed during the build?

No response

Tool installation time in runtime

No response

Are you willing to submit a PR?

I can try...

[miketimofeev]

Hi @bwoodsend!
Just to clarify what we have on the image:

• Python3 installed from homebrew https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/python@3.9.rb
• Pre-cached python versions built from source using this repository https://github.com/actions/python-versions to use with https://github.com/actions/setup-python

We can't do anything with the one from brew but we can probably try to re-build the pre-cached one with the arm64 support. Unfortunately, it may take some time so if you have a chance you can submit a PR to the python-version repo modifying the macos-python-builder script https://github.com/actions/python-versions/blob/main/builders/macos-python-builder.psm1

[bwoodsend]

OK, I'll do a PR for option 2. The python-versions source code looks a lot less intimidating than I expected so this shouldn't be too hard.

Do you have any thoughts on whether all Python versions that are capable of universal2 builds should become universal2 or should it be opt-in via a universal2: true flag to actions/setup-python? I can't think of any reason why a universal2 build would impair someone's workflow. They have a bigger filesize but that shouldn't slow builds down because these Python installers are cached I believe so the VMs don't even have to download them?

[miketimofeev]

What is the difference in size? If it's just a few megabytes then we should be good to go with the universal by default

[maxim-lobanov]

We would avoid adding additional parameters on actions/setup-python side so rebuilding existing binaries with universal2 support is more preferable. Of course, in case if it is not a breaking changes

[bwoodsend]

They're effectively two copies in one so the binaries about double the size of a single arch Python. But they compress down well when you archive them.

	single	universal2
unpacked	161MB	215MB
archive	29MB	36MB