Description
macOS supports multi-architecture binaries (fat binaries) and CPython have long supported building them. Current releases of macOS support two system architectures: legacy x86_64 (for Intel Macs) and arm64 (for Apple Silicon Macs). The CPython ./configure supports building fat binaries containing both archs by use of the --with-universal-archs=universal2 option. To do so, it makes use of the muilt-arch support built into the Apple-supplied build tools (Xcode or Command Line Tools) which allows multi-arch compiles and links to happen via single calls (by adding --arch flags to the calls to compilers et al).
However, because configure's autoconf tests are only run in the architecture of the build machine, different results can be obtained depending on whether the universal2 build machine is an Intel or an Apple Silicon Mac. In particular, there are differences in the autoconf generated pyconfig.h as shown here with modified file names for clarity; in either case, there is only one pyconfig.h file that will be generated and used regardless of which arch the fat binaries are run on:
$ diff pyconfig_intel.h pyconfig_arm.h
24c24
< #define ALIGNOF_MAX_ALIGN_T 16
---
> #define ALIGNOF_MAX_ALIGN_T 8
439c439
< #define HAVE_GCC_ASM_FOR_X64 1
---
> /* #undef HAVE_GCC_ASM_FOR_X64 */
443c443
< #define HAVE_GCC_ASM_FOR_X87 1
---
> /* #undef HAVE_GCC_ASM_FOR_X87 */
1607c1607
< #define PY_SUPPORT_TIER 1
---
> #define PY_SUPPORT_TIER 2
1656c1656
< #define SIZEOF_LONG_DOUBLE 16
---
> #define SIZEOF_LONG_DOUBLE 8
The difference in ALIGNOF_MAX_ALIGN_T between the two archs leads to the following test failure when a universal2 binary is built on an Apple Silicon Mac but executed on an Intel Mac:
test_alignof_max_align_t (test.test_capi.test_misc.Test_testcapi.test_alignof_max_align_t) ... Assertion failed: (ALIGNOF_MAX_ALIGN_T >= _Alignof(long double)), function test_alignof_max_align_t, file heaptype_relative.c, line 309.
Fatal Python error: Aborted
The test does not crash in the opposite case: when built on an Intel Mac but run on an Apple Silicon Mac, because in this case 16 > 8.
This issue was noticed while testing changes to the process we use to build python.org python for macOS installers. Up to now, we have been using Intel Macs to build all installers but, when testing building installers on Apple Silicon Macs, the crash of test_capi was seen when running the resultant universal2 installer on Intel Macs.
It's not clear what is the best way to resolve this and, more importantly, whether there are other kinds of similar problems not yet discovered. Probably the safest approach would be to build each architecture separately on their native archs and lipo the resultant single-arch binaries into fat files. And/or it may be better to provide separate copies of arch-sensitive header files like pyconfig.h in separate directories and have the interpreter point to the correct one at run time for extension module builds. Or, in this case, perhaps adding some some conditional code to force the sizes to be the larger value in the case of macOS universal2 builds might be enough this time?