Progress to TenFourFox 45: milestone 1

TenFourFox 40 is a thing, but changesets only, available from SourceForge. It starts up, passes the JavaScript test suite, does browsery things and doesn't crash. Unfortunately 40 took longer than I had planned to get off the ground and I think we'll have to do a "38.9" to buy us one more cycle; in the meantime, I've decided to skip a step by jumping directly to 43 and cross my fingers that it works. If I find a serious regression, I'll have to decide whether I want to back up to 41 or 42. If not, the third milestone will be the first 45 beta.

Builders take note that MacPorts gcc 4.8 is now required; gcc 4.6 will no longer build the browser and even adding back the compatibility shim from 38 will probably not be sufficient (not only will it probably not compile, it won't link either due to required C++11 atomics support). However, I'd still like to get other compiler choices available since MacPorts is kind of a rickety platform base and always subject to some degree of breakage. If people try Sevan's pkgsrc gcc or Misty's Tigerbrew gcc, I'd like to hear your comments about how well they functioned and/or what was needed to get them working.

The practicality of the case for TenSixFox

After Mozilla's announcement that 10.6-10.8 support is ending, Dan Knight posted two articles on LowEndMac regarding whither a "TenSixFox," both for and against.

TenFourFox gets cited a lot as a successful example of a community-driven Firefox fork (to Tiger PowerPC, for those on our syndicated sites who are unfamiliar), and we're usually what gets pointed to when some open-source project (including Firefox) stops supporting a platform. However, even though we have six years so far of successful ports in the can, we're probably not a great example anymore. While we've had major contributors in the past -- Tobias and Ben in particular, but others as well -- and Chris T still volunteers his time in user support and herds the localizer cats, pretty much all the code and porting work is done entirely by yours truly these days. It's just not the community it was, and it's a lot of work for me personally. Nevertheless, our longevity means the concept of a "TenSixFox" gets brought up periodically with expectations of the same success, as Dan did in his first piece, and as a result a lot of people assume that our kind of lightning will simply strike twice.

Unfortunately, even if you found someone as certifiably insane highly motivated as your humble narrator willing to do it, the tl;dr is that a TenSixFox can't be accomplished in the same way TenFourFox was. Restoring 10.4 compatibility to Firefox is, at the 50,000' level, a matter of "merely" adding back the code Mozilla removed to the widget and graphics modules (primarily between Firefox 3.6 and 4, though older code has been gradually purged since then). The one thing that could have been problematic in the beginning was font support, since Tiger's CoreText is inadequate for handling text runs of the complexity required and the ATSUI support had too many bugs to keep it working, but Mozilla solved that problem for us by embedding a font shaper for its own use (Harfbuzz) which we adopted exclusively. After that, as versions ticked along, whatever new features they introduced that we couldn't support we just ruthlessly disabled (WebGL, WebRTC, asm.js, etc.), but the "core" of the browser, the ability to interpret JavaScript, display HTML and handle basic chrome, continued to function. Electrolysis fundamentally changes all of that by introducing a major, incompatible and (soon to be) mandatory change to the browser's basic architecture. It already has serious issues on 10.6 and there is nothing that says those issues are fixable or avoidable; it may well be impossible to get multi-process operational on Snow Leopard without significant showstoppers. That's not a recipe for continued functionality, and there is no practical way of maintaining single-process support once it disappears because of how pervasive the changes required will eventually be.

Plus, there's another technical problem as well. Currently Firefox can't even be built on 10.6; as of this writing the minimum supported compiler is clang 4.2, which means Xcode 4.6 at minimum, which requires 10.8. This isn't a showstopper yet because cross-building is a viable if inconvenient option, but it makes debugging harder, and since 10.8 is being dropped too it is probable that the Xcode minimum requirement will jump as well.

So if you're that nuts enthusiastic, and you have the chops, what are your options?

Option 1 is to keep using TenFourFox on Rosetta, with all the performance and functionality limitations that would imply. Let me reiterate that I don't support Intel Macs and I still have no plans to; this project is first and foremost for Power Macs. I know some of you do run it on 10.6 for yuks and some of you even do so preferentially, but there's not going to be another Intel build of TenFourFox unless someone wants to do it themselves. That brings us to ...

Option 2 is to build TenFourFox natively for 10.6. I currently only support linking against the 10.4 SDK because of all the problems we had trying to maintain a codebase linkable against 10.4 and 10.5 (Tobias used to do this for AuroraFox but that support is long gone). Fortunately, Xcode 3 will still allow this and you can build and debug directly on a Snow Leopard system; plus, since the system minimum will be 10.6 anyway, you'll dodge issue 209 and you have the option of 10.5 compatibility too. You'll have to get all the MacPorts prerequisites including gcc 4.8, enable the x86 JIT again in JavaScript and deal with some of our big-endian code for fast font table enumeration and a couple other things (the compiler will throw an error so you'll know where to make the changes), but the rest of it should "just work." The downside is that, since you are also linking against the 10.4 SDK (regardless of your declared minimum OS support), you'll also have all the limitations of TenFourFox because you're actually porting it, not Firefox (no WebGL, no plugins, no WebRTC, no CoreText shaper; and, because we use a hybrid of Carbon and Cocoa code, you're limited to a 32-bit build). On the other hand, you'll get access to our security updates and core improvements, which I plan to keep making for as long as I personally use TenFourFox, so certain things will be more advanced than the next option(s), which are ...

Option 3 is to keep building 45ESR for 10.6 after 45ESR support terminates by continuing to port security updates. You won't need to make any changes to the code to get the build off the ground, but you'll need both 10.6 and 10.8 systems to debug and build respectively, and every six weeks you'll have to raid the current ESR branch and backport the changes. You won't get any of the TenFourFox patches and core updates unless you manually add them yourself (and debug them, since we won't support you directly). However, you'll have the option of a 64-bit build and you'll have all the supported 10.6 features as well as support for 10.7 and 10.8.

Finally, option 4 is to keep building firefox-release after 45ESR for 10.6 until it doesn't work anymore. Again, you'll need both 10.6 and 10.8 systems to be effective. This option would get you the most current browser in terms of technology, and even if Electrolysis is currently the default, as long as single-process mode is still available (such as in Safe Mode or for accessibility) you should be able to hack the core to always fall back to it. However, now that 10.6-10.8 support is being dropped, you'll have to undo any checks to prevent building and running on those versions and you might not get many more release cycles before other things break or fail to build. Worst of all, once you get marooned on a particular version between ESRs you'll find backporting security patches harder since you're on a branch that's neither fish nor fowl (from personal experience).

That's about the size of it, and I hope it makes it clear TenFourFox and any putative TenSixFox will exist in very different technological contexts. By the time 45ESR support ends, TenFourFox will have maintained source parity with Firefox for almost seven years and kept up with most of Gecko's major technology advancements, and even the very last G5 to roll off the assembly line will have had 11 years of support. I'm pretty proud of that, and I feel justifiably so. Plus, my commitment to the platform won't end there when we go to feature parity; we'll be keeping a core that can still handle most current sites for at least a couple years to come, because I'll be still using it.

TenSixFox, on the other hand, even if it ends up existing, will lack source parity with Firefox just about from its very beginning. It's not going to be the technological victory that we were at our own inception in 2010, and it's not likely to be the success story we've been for that and other important reasons. That doesn't mean it's not worth it, just that there will be some significant compromises and potential problems, and you'd have to do a substantial amount of work now and in the future to keep it viable. For many of you, that may not be worth it, but for a couple of you, it just might be.

Après moi, le Tier-3 holocaust

Well, it finally happened: Mozilla's going to end support for 10.6-10.8. Actually, I'm surprised it took this long:

The motivation for this change is that we have continued failures that are specific to these old operating systems and don't have the resources on engineering teams to prioritize these bugs. Especially with the deployment of e10s we're seeing intermittent and permanently [sic] failures on MacOS 10.6 that we are not seeing elsewhere.

This confirms my suspicions that Electrolysis would have been seriously problematic on 10.4 and 10.5 given the permafails Mozilla is seeing on 10.6, even if we got it to compile (it doesn't even do that on Tiger yet). That's why we're forking after 45ESR, which doesn't have E10S enabled by default.

Interestingly, Mozilla's plan is nearly identical to ours: move 10.6-10.8 users onto 45ESR as well so that they'll still get a year-ish of support on that branch in a sort of graceful wind-down. The difference is that we plan to make hacks to the core to support certain post-45 features (making TenFourFox essentially into an OS X Classilla), and of course backport future security updates from 52ESR and so on. I guess if you're running Snow Leopard you can use TenFourFox post-fork with the usual limitations, but I still currently have no plans for an Intel-native version unless someone steps up to maintain it (although I have a 10.6 machine that might benefit, I just don't have the time right now).

The interesting part is that Mozilla still has more users on 10.6 than 10.7 and 10.8 combined, and 10.6 still accounts for something like 13% of all Firefox Mac users. As late as 2014 19% of all Macs ran it. Snow Leopard really is the Windows XP of the Intel Mac.

Mozilla also did a wholesale pruning of most of the other Tier-3 ports, as part of shifting from autoconf:

Following our official move off autoconf, the core build team is faced with having to convert more than 17k lines of shell+m4. A large part of those are to support Tier-3 platforms such as Solaris, HPUX, AIX, etc., with compilers that are not MSVC, GCC or clang (e.g. SunPro, XL C++...). To simplify the conversion, from day one, we will support only the following set of platforms and toolchains:

Platforms:
- Linux
- Android
- Windows
- OSX/Darwin
- multiple flavors of BSD (kFreeBSD, FreeBSD, DragonFly, NetBSD, OpenBSD)

Toolchains:
- MSVC
- GCC/mingw
- clang
- clang-cl

This is a little different than the end of 10.[678] support, however:

This does not mean Firefox/Gecko/etc. will forever drop support for these platforms and toolchains. It simply means we can't assess what parts of the shell+m4 are relevant (as opposed to cruft accumulated over years or decades) and will continue to work (since we don't have automation to verify these configurations).

[...]

The good news is that dropping support now will help make the overall conversion happen sooner, *and*, once the conversion is done, interested parties can come back with working patches that should be easier to write.

My SPARC Ultra-3 laptop (a rebadged Tadpole Viper) runs an ESR build of Firefox just fine in Solaris 10, the last Solaris supported on that platform. Unfortunately I don't have the Sun compiler, or I'd take a whack at building it myself. At least someone out there is still popping out contrib builds for SPARC and i386, but I don't know if they will continue to.

On the other hand, some of those other platforms haven't worked in years. I know of a Firefox 3.6 for AIX, at least on POWER (my PowerPC AIX systems barf on it), but HP/sUX ended support somewhere around 3.5.9 and was always a mess to build (I hated HP's ANSI C-compiler back in the day when I had to administer those systems). I even remember seeing some Digital UNIX/Tru64 stuff deep within xpcom/, but I can't imagine full support persisted much beyond Mozilla 1.8 something. None of the rest of them got much further.

Really, these are all dusty legacies of how portable the old codebase used to be. At one time, Netscape ran on Windows (as early as 3.1), Mac OS (System 7 and up), OS/2, Linux, Digital UNIX (Tru64), SunOS 4, VMS (VAX and Alpha), Solaris, BSD, HP/UX, IRIX, AIX, and probably some other minority ports I don't remember, and all with almost total feature parity. In fact, I myself have personally used every single one of those ports at one time or another. As late as Mozilla 1.7 almost all of those platforms were still working (SunOS 4 and VMS were gone, and Mac OS was replaced with OS X, but the rest were still functioning), but Firefox 2 and 3 gradually winnowed the rest, and by Firefox 4 it was pretty much just Windows, OS X (including us), Linux, the BSDs and Solaris. Soon it'll just be those OSes on x86-64 and ARM.

So, while it may have been hell to maintain, that's a lot of history gone that I personally lived through and good cause or not it still makes me sad to see these last remnants drift away, even on the abandoned platforms. (You can cram it if you're going to post some uninsightful comment like "paying back technological debt" or "no one maintains platforms for free." I'm well aware of the cost of cruft. That doesn't mean it doesn't have historical value, and that historical value should be appreciated, even if doing so is best done in archives rather than current code bases.) It used to be neat to compare how heterogeneous and diverse computing platforms were back in the day, but sadly today's platforms are more alike than they're different, and I think we've lost something there.

It's not really fun anymore.

38.7.0 available (plus: thanks, Mozilla, for making the web little-endian)

As I try to recover from my annual bout of viral bronchitis, TenFourFox 38.7.0 is now available for testing (downloads, release notes, hashes). Sharp-eyed builders will note that the hash for the 38.7.0 changesets is the same as the one for the 38.6.1 changesets, and that's because no changes took place between 38.6.1 and 38.7.0, so they're identical. That was good because I could just let the automated patchers build and package this version and I could sleep most of the last two days. As usual, it will become official next Monday evening (more or less), Pacific time.

I finished my Master's coursework on Monday, so now the assault on 45ESR begins in earnest. I forecast about a 70% chance of success since we have a working gcc 4.8 from MacPorts which built 38 successfully and Electrolysis and Rust are not mandatory in 45. 45.0 comes out with 38.7 next week, 45.1 comes out on 19 April (with 38.8) and 45.2 comes out on 7 June, marking the end of support for 38ESR. However, I need to have basic working versions of 40, 41, 42 and 43 first, which I already have local trees for, so I can do regression testing on bugs that crop up in the testing phase.

Ordinarily that would still be ample time since this is more or less a straight port. The complicating factor is me getting married -- yes, folks, I'm off the market to a lovely Australian lady -- which will take about four weeks out of the eight to ten weeks I'll have to complete it (there's a total of twelve-ish weeks in there but I'd like at least two weeks for the localization and for you lot to test it, and even that will be tight), so if needed there will be an unofficial 38.9 to buy us another six by backporting the security patches for 45.2.

Current plans for 45 are to add basic user-agent switching support to the core browser (since TenFourFox will fork with the end of 45ESR support) and expose turning off PDF.js in the UI. Under the hood, I would like to implement some PDF.js fixes and implement a true JavaScript Intl library instead of the hacky emulation we have now by getting ICU working properly. I might also do something about bringing a gimped newtab page back since a lot of people missed that functionality in 38. These changes can be introduced later in the 45 time frame and are not necessary for launch, but the strings will be present in advance in a special tenfourfox.dtd so that localizers can have them ready by the time the features are introduced.

In addition, I've been working on spec plans for IonPower-NVLE (Non-Volatile Little-Endian), which is a moderate-scope refactor of our IonPower PowerPC JavaScript JIT compiler to ("NV") make the IonMonkey allocator only use the ABI non-volatile registers, and to ("LE") make all typed-array access little endian using the standard PowerPC byteswapped load and store instructions (we already use these instructions in the big-endian port of irregexp, so they are well-tested for our purposes). The NV portion reflects that our code generator basically follows its own ABI rules within Ion-generated code but has to thunk back to the OS when calling library routines, which requires saving and restoring all the volatile registers since the Ion code generators generally use those first. However, we have substantially more registers than any of the Tier-1 platforms, so we can do better. By forcing the IonMonkey allocator only to use the declared non-volatile registers, we only have to save and restore them in the trampoline when Ion code is called initially, substantially reducing the overhead of OS calls -- especially within code generated by the Baseline compiler, which we are tuned to enter relatively earlier than other platforms. Plus, with volatile registers now being freed between calls and no longer known to IonMonkey, we can use them with few restrictions for better ILP in complex local code blocks we emit internally from the portions of the JIT we completely control.

On the other hand, the reason why we need the LE portion is significantly more frustrating philosophically, even though it seems simple. Thanks to asm.js and its byte-level access using typed arrays, lots of sites like Faceblech and WhatsCrap are Emscriptening their way to performance by turning a little-endian binary into (fast) little-endian JavaScript. Even if the library or code block they were compiling can be built big-endian, since it's invariably being built on a little-endian x86 system Emscripten dutifully poops out little-endian asm.js on the other end and that's what we end up (unsuccessfully) executing. We've already gotten bitten by this in several places and it is reasonable to expect it will happen more and more often; Tobias proved that the endianness of the code was responsible by doing an analogous change for Leopard WebKit which fixed the problem and we need to do the same. I will add typed-array byteswapping to the interpreter first and get that working against the test suite and the core browser, and then add it to the JIT. As a side effect, since we are now effectively little-endian as far as JavaScript applications are concerned, it may be worth looking into the feasibility of native asm.js on PowerPC again later down the road.

But I'm pretty irked that the upshot of all this is a little-endian web. We can't rely on even non-WebGL JavaScript to be system-agnostic anymore and that seems like a pretty badly broken promise about how architecture-independent the Web was supposed to be. I realize this outcome wasn't at all the intention when Mozilla introduced it, but by making simple tools to turn compiler output into ready-to-use asm.js that's what was inevitably going to happen, and now that developers are finding asm.js too tempting to ignore, that's exactly what is happening. Overall, minority platforms like ours are about to get even more marginalized on the Web without rather drastic steps like this one (Firefox on PowerPC/SPARC Linux and BSD, take note), so look for testing builds with IonPower-NVLE probably by the end of this year.

That said, that sexy POWER8 workstation can run little-endian. I'm just saying.