Wednesday, June 24, 2020

The Super Duper Universal Binary

A question I got repeatedly the last couple days was, now that AARM (Apple ARM) is a thing, is the ultimate ARM-Intel-PowerPC Universal Binary possible? You bet it is! In fact, Apple already documents that you could have a five-way binary, i.e., ARM64, 32-bit PowerPC, 64-bit PowerPC, i386 and x86_64. Just build them separately and lipo them together.

But it's actually more amazing than that because you can have multiple subtypes. Besides generic PPC or PPC64, you can have binaries that run specifically on the G3 (ppc750), G4 (ppc7400 or ppc7450) or G5 (ppc970). The G5 subtype in particular can be 32-bit or 64-bit. I know this is possible because LAMEVMX is already a three-headed binary that selects the non-SIMD G3, AltiVec G4 or special superduper AltiVec G5 version at runtime from a single file. The main reason I don't do this in TenFourFox is that the resulting executable would be ginormous (as in over 500MB in size).

But ARM has an even more dizzying array of subtypes, at least nine, and the Apple ARM in the new AARM Macs will probably be a special subtype of its own. This means that theoretically a Super Duper Universal Binary ("SDUB") could have all of the following:

  • ppc750
  • ppc7400
  • ppc7450
  • ppc970 (this would work for both 32-bit and 64-bit on the G5)
  • i386
  • x86_64
  • x86_64h (i.e., Haswell, here's an example, thanks Markus Stange for pointing this out)
  • armv4t
  • armv5
  • armv6
  • armv6m
  • armv7
  • armv7em
  • armv7k
  • armv7m
  • armv7s
  • whatever AARM Macs turn out to be

That's potentially a 17-way binary. The best part is that each individual subpart can link against a different Mac OS X/macOS SDK, so the PowerPC subportions could be linked against 10.4, the i386 subportion against anything from 10.4 through 10.6, the x86_64 subportion against 10.4 through 10.15 (+/- the Haswell subtype), the various weirdo ARM subportions against whatever macOS SDK is relevant to the corresponding iOS version, and the AARM Mac-specific subportion against 11.0. It may be necessary to lipo it together in multiple stages using multiple machines or Xcodes depending on which subtypes are known to that platform, but after you do that, code-sign and/or notarize you should have the ultimate Super Duper Universal Binary able to run on any of these systems. Now, there's a challenge for someone. I look forward to one of those Developer Transition Kits getting put to good use here.

12 comments:

  1. Back in the early days of OS X, there were "universal" binaries that ran natively in both OS X and Classic mode, and before that there were "universal" binaries (although I'm not sure they were called that) that included both m68k and PPC instructions. I once encountered an app that ran on anything from System 7 to OS X by glomming together different binaries made with different toolchains, so I can but hope that someday somebody will make at least a Hello World that runs on m68k/PPC/Intel/ARM.

    ReplyDelete
    Replies
    1. 68K and PPC binaries were called "fat." I guess that's not politically correct anymore. :)

      You're thinking of Carbon apps. Mac OS 9 cannot run Mach-O binaries, which are the ones we're talking about in this post. However, Mac OS X can run CFM executables, or at least they could until the end of Rosetta. CFM executables can only contain 68K and PowerPC code (other architectures are not supported), and since there was never a CarbonLib for 68K, there would be no way to do this at least with Mac OS.

      On NeXTSTEP, you could have a 68K/Intel/SPARC/PA-RISC binary, and Rhapsody added PPC, but Mac OS X can't run NeXT apps generally.

      Delete
    2. (The trick you may have seen is using application bundles to run different executables depending on the OS. That's a little exotic and wouldn't strictly speaking be a "Universal" binary even though it would have tha teffect.)

      Delete
  2. There's also x86_64h, the "Haswell" subtype of x86_64.

    ReplyDelete
  3. Wow. This is amazing.

    I, for one, would love to see again the old UNIX days, with many different platforms trying to innovate and move the world forward in a different way.

    So many good ideas that could see a lovely outcome with modern fabs.

    ReplyDelete
  4. MacPorts (which has extremely strong support for 10.4 and 10.5 Intel and PowerPC Macs, BTW) has an automatic universal build mode called the "muniversal PortGroup". You feed it a list of architectures, and it generates a separate build folder for each one, running configure and build with the architectures you want.

    Then at the end of the builds, it automatically lipos all the binaries together into on universal set of binaries.

    The only hiccup comes with some software like libraries when generated text files, like the pkgconfig files, are different, and then a plan needs to be made on how to manage that, but even most of that is automatic.

    Should try it!

    ReplyDelete
  5. Anyone know if Apple has any special features in the binaries that help the the binary translation from X86 to ARM?

    ReplyDelete
    Replies
    1. No, but I would not be surprised if AARM has some special instructions to improve R2's performance.

      Delete
  6. Wait until you see the full contents of /usr/include/mach/machine.h ;)

    ReplyDelete
    Replies
    1. (Checks the copy shipped with 10.11.6.) Holy cow, that has to be every architecture ever recognized by XNU…!

      Delete
  7. Your article even made it to "the" German news site about IT technology. Heise is called Uncle Heinz in Germany.
    https://www.heise.de/news/macOS-Universal-Binaries-mit-ARM-Intel-und-PowerPC-4842217.html

    Congrats ;-)

    ReplyDelete

Due to an increased frequency of spam, comments are now subject to moderation.