Thursday, June 23, 2016

Progress to TenFourFox 45: milestone 2 (plus: get your TALOS on or else, and Let's Engulf Comodo)

After a highly prolonged porting phase, TenFourFox 43 finally starts up, does browsery things and doesn't suck. Changesets are available on SourceForge for your amusement. I am not entirely happy with this release, though; our PowerPC JavaScript JIT required substantial revision, uncovering another bug which will be fixed in 38.10 (this one is an edge case but it's still wrong), and there is some glitch in libnestegg that keeps returning the audio sampling rate on many WebM videos as 0.000. I don't think this is an endian problem because some videos do play and I can't figure out if this is a legitimate bug or a compiler freakout, so right now there is a kludge to assume 22.050kHz when that happens. The video is otherwise parseable and that gets it to play, but I find this solution technically disgusting, so I'm going to ponder it some more in the meantime. We'll see if it persists in 45. On the other hand, we're also able to load the JavaScript Internationalization API now instead of using our compat shim, which should fix several bugs and add-on compatibility issues.

Anyway, the next step is to port 45 using the 43 sets, and that's what I'll be working on over the next several weeks. I'm aiming for the first beta in mid-July, so stay tuned.

For those of you who have been following the Talos POWER8 workstation project (the most powerful and open workstation-class Power Architecture system to date; more info here and here), my contacts inform me that the fish-or-cut-bait deadline is approaching where Raptor needs to determine if the project is financially viable with the interest level so far received. Do not deny me my chance to give them my money for the two machines I am budgeting (a kidneystone) for. Do not foresake me, O my audience. I will find thee and smite thee. Sign up, thou cowards, and make this project a reality. Let's get that Intel crap you don't actually control off thy desks. You can also check out using the Talos to run x86 applications through QEMU, making it the best of both worlds, as demonstrated by a video on their Talos pre-release page.

Last but not least, increasingly sketchy certificate authority and issuer Comodo, already somewhat of a pariah for previously dropping their shorts, has decided to go full scumbag and is trying to trademark "Let's Encrypt." Does that phrase seem familiar to you? It should, because "Let's Encrypt" is (and has been for some time) a Mozilla-sponsored free and automated certificate authority trying to get certificates in the hands of more people so that more websites can be protected by strong encryption. As their FAQ says, "Anyone who owns a domain name can use Let's Encrypt to obtain a trusted certificate at zero cost."

Methinks Comodo is hoping to lawyer Let's Encrypt out of existence because they believe a free certificate issuer will be a huge impact to their business model. Well, yes, that's probably true, which makes me wonder what would happen if Mozilla threatened to pull the Comodo CA root out of Firefox in response. Besides, based on this petulant and almost certainly specious legal action and their previous poor security history, the certificate authority pool could definitely use a little chlorine anyhow.

Saturday, June 4, 2016

38.9 available

TenFourFox 38.9 is available for testing (downloads, hashes, release notes). This includes the IonPower JavaScript JIT fix in issue 319 and relevant backported security patches from Firefox ESR 45.2 enumerated in issue 320. If you're a downstream builder hoping to use our backports to keep your own 38ESR fork alive, keep in mind I've only imported the security patches that are actually significant to us (for example, we run almost none of the OpenGL code, so OpenGL and WebGL patches are invariably irrelevant to TenFourFox and I hardly ever import them), so you should investigate all the other 45ESR security advisories as well to make sure they do not also apply to you.

Since I know Ric Ford from MacInTouch reads this blog for updates (hi Ric!), let me also be clear that 38.9 is an interim stopgap release only -- the plan is still to get TenFourFox to version 45, hopefully by 45.4. There will also likely be a 38.10 (security parity with official Firefox ESR 45.3) to allow 45 enough time to bake in beta and the localizers to catch up, but I'm determined to get us there ultimately with more information on my plans to come. 38.9 and 38.10 are TenFourFox-specific updates to Firefox 38ESR and do not correspond to any official version of Firefox.

38.9 will become live on Monday evening Pacific as usual, assuming no showstoppers.

Tuesday, May 31, 2016

Extending TenFourFox 38

First, to satisfy your curiosity:

Yes, it was a lovely wedding. Thanks for your well-wishes.

I've been slogging through the TenFourFox 43.0 port and while the JIT is coming along slowly and I'm making bug fixes as I find them, I still have a few critical failures to smoke out and I'd say I'm still about two weeks away from tackling 45. Thus, we'll definitely have an TenFourFox 38.9 consisting of high-priority cross-platform security fixes backported from Firefox 45ESR and there may need to be a 38.10 while TenFourFox 45 stabilizes. This 38.9, which I remind you does not correlate to any official Firefox version, will come out parallel to 45.3 and I should have test builds by this weekend for the scheduled Tuesday release. TenFourFox 38.9 will also include a fix for a JIT bug that I discovered in 43 that also affects 38.

I'm also accumulating post-45 features I want to add to our fork of Gecko 45 to keep it relevant for a bit longer. Once 45 gets off the ground, I'm considering having an unstable and stable branch again with features moving from unstable to stable as they bake in beta testing, though unlike Mozilla our cycles will probably be 12 weeks instead of 6 due to our smaller user base. These features are intentionally ones that are straightforward to port; I'm not going to do massive surgery like what was required for Classilla to move forward. I definitely learned my lesson about that.

More later.

Friday, April 22, 2016

38.8.0 available

38.8.0 is available (downloads, hashes, release notes). There are no major changes, only a bustage fix for one of the security updates that does not compile under gcc 4.6. Although I built the browser and did all the usual custodial tasks remotely from a hotel room in Sydney, assuming no major showstoppers I will actually take a couple minutes on my honeymoon to flip the version indicator Monday Pacific time (and, in a good sign for the marriage, she accepts this as a necessary task).

Don't bother me on my honeymoon.

Thursday, March 31, 2016

Progress to TenFourFox 45: milestone 1

TenFourFox 40 is a thing, but changesets only, available from SourceForge. It starts up, passes the JavaScript test suite, does browsery things and doesn't crash. Unfortunately 40 took longer than I had planned to get off the ground and I think we'll have to do a "38.9" to buy us one more cycle; in the meantime, I've decided to skip a step by jumping directly to 43 and cross my fingers that it works. If I find a serious regression, I'll have to decide whether I want to back up to 41 or 42. If not, the third milestone will be the first 45 beta.

Builders take note that MacPorts gcc 4.8 is now required; gcc 4.6 will no longer build the browser and even adding back the compatibility shim from 38 will probably not be sufficient (not only will it probably not compile, it won't link either due to required C++11 atomics support). However, I'd still like to get other compiler choices available since MacPorts is kind of a rickety platform base and always subject to some degree of breakage. If people try Sevan's pkgsrc gcc or Misty's Tigerbrew gcc, I'd like to hear your comments about how well they functioned and/or what was needed to get them working.

Saturday, March 19, 2016

The practicality of the case for TenSixFox

After Mozilla's announcement that 10.6-10.8 support is ending, Dan Knight posted two articles on LowEndMac regarding whither a "TenSixFox," both for and against.

TenFourFox gets cited a lot as a successful example of a community-driven Firefox fork (to Tiger PowerPC, for those on our syndicated sites who are unfamiliar), and we're usually what gets pointed to when some open-source project (including Firefox) stops supporting a platform. However, even though we have six years so far of successful ports in the can, we're probably not a great example anymore. While we've had major contributors in the past -- Tobias and Ben in particular, but others as well -- and Chris T still volunteers his time in user support and herds the localizer cats, pretty much all the code and porting work is done entirely by yours truly these days. It's just not the community it was, and it's a lot of work for me personally. Nevertheless, our longevity means the concept of a "TenSixFox" gets brought up periodically with expectations of the same success, as Dan did in his first piece, and as a result a lot of people assume that our kind of lightning will simply strike twice.

Unfortunately, even if you found someone as certifiably insane highly motivated as your humble narrator willing to do it, the tl;dr is that a TenSixFox can't be accomplished in the same way TenFourFox was. Restoring 10.4 compatibility to Firefox is, at the 50,000' level, a matter of "merely" adding back the code Mozilla removed to the widget and graphics modules (primarily between Firefox 3.6 and 4, though older code has been gradually purged since then). The one thing that could have been problematic in the beginning was font support, since Tiger's CoreText is inadequate for handling text runs of the complexity required and the ATSUI support had too many bugs to keep it working, but Mozilla solved that problem for us by embedding a font shaper for its own use (Harfbuzz) which we adopted exclusively. After that, as versions ticked along, whatever new features they introduced that we couldn't support we just ruthlessly disabled (WebGL, WebRTC, asm.js, etc.), but the "core" of the browser, the ability to interpret JavaScript, display HTML and handle basic chrome, continued to function. Electrolysis fundamentally changes all of that by introducing a major, incompatible and (soon to be) mandatory change to the browser's basic architecture. It already has serious issues on 10.6 and there is nothing that says those issues are fixable or avoidable; it may well be impossible to get multi-process operational on Snow Leopard without significant showstoppers. That's not a recipe for continued functionality, and there is no practical way of maintaining single-process support once it disappears because of how pervasive the changes required will eventually be.

Plus, there's another technical problem as well. Currently Firefox can't even be built on 10.6; as of this writing the minimum supported compiler is clang 4.2, which means Xcode 4.6 at minimum, which requires 10.8. This isn't a showstopper yet because cross-building is a viable if inconvenient option, but it makes debugging harder, and since 10.8 is being dropped too it is probable that the Xcode minimum requirement will jump as well.

So if you're that nuts enthusiastic, and you have the chops, what are your options?

Option 1 is to keep using TenFourFox on Rosetta, with all the performance and functionality limitations that would imply. Let me reiterate that I don't support Intel Macs and I still have no plans to; this project is first and foremost for Power Macs. I know some of you do run it on 10.6 for yuks and some of you even do so preferentially, but there's not going to be another Intel build of TenFourFox unless someone wants to do it themselves. That brings us to ...

Option 2 is to build TenFourFox natively for 10.6. I currently only support linking against the 10.4 SDK because of all the problems we had trying to maintain a codebase linkable against 10.4 and 10.5 (Tobias used to do this for AuroraFox but that support is long gone). Fortunately, Xcode 3 will still allow this and you can build and debug directly on a Snow Leopard system; plus, since the system minimum will be 10.6 anyway, you'll dodge issue 209 and you have the option of 10.5 compatibility too. You'll have to get all the MacPorts prerequisites including gcc 4.8, enable the x86 JIT again in JavaScript and deal with some of our big-endian code for fast font table enumeration and a couple other things (the compiler will throw an error so you'll know where to make the changes), but the rest of it should "just work." The downside is that, since you are also linking against the 10.4 SDK (regardless of your declared minimum OS support), you'll also have all the limitations of TenFourFox because you're actually porting it, not Firefox (no WebGL, no plugins, no WebRTC, no CoreText shaper; and, because we use a hybrid of Carbon and Cocoa code, you're limited to a 32-bit build). On the other hand, you'll get access to our security updates and core improvements, which I plan to keep making for as long as I personally use TenFourFox, so certain things will be more advanced than the next option(s), which are ...

Option 3 is to keep building 45ESR for 10.6 after 45ESR support terminates by continuing to port security updates. You won't need to make any changes to the code to get the build off the ground, but you'll need both 10.6 and 10.8 systems to debug and build respectively, and every six weeks you'll have to raid the current ESR branch and backport the changes. You won't get any of the TenFourFox patches and core updates unless you manually add them yourself (and debug them, since we won't support you directly). However, you'll have the option of a 64-bit build and you'll have all the supported 10.6 features as well as support for 10.7 and 10.8.

Finally, option 4 is to keep building firefox-release after 45ESR for 10.6 until it doesn't work anymore. Again, you'll need both 10.6 and 10.8 systems to be effective. This option would get you the most current browser in terms of technology, and even if Electrolysis is currently the default, as long as single-process mode is still available (such as in Safe Mode or for accessibility) you should be able to hack the core to always fall back to it. However, now that 10.6-10.8 support is being dropped, you'll have to undo any checks to prevent building and running on those versions and you might not get many more release cycles before other things break or fail to build. Worst of all, once you get marooned on a particular version between ESRs you'll find backporting security patches harder since you're on a branch that's neither fish nor fowl (from personal experience).

That's about the size of it, and I hope it makes it clear TenFourFox and any putative TenSixFox will exist in very different technological contexts. By the time 45ESR support ends, TenFourFox will have maintained source parity with Firefox for almost seven years and kept up with most of Gecko's major technology advancements, and even the very last G5 to roll off the assembly line will have had 11 years of support. I'm pretty proud of that, and I feel justifiably so. Plus, my commitment to the platform won't end there when we go to feature parity; we'll be keeping a core that can still handle most current sites for at least a couple years to come, because I'll be still using it.

TenSixFox, on the other hand, even if it ends up existing, will lack source parity with Firefox just about from its very beginning. It's not going to be the technological victory that we were at our own inception in 2010, and it's not likely to be the success story we've been for that and other important reasons. That doesn't mean it's not worth it, just that there will be some significant compromises and potential problems, and you'd have to do a substantial amount of work now and in the future to keep it viable. For many of you, that may not be worth it, but for a couple of you, it just might be.

Wednesday, March 16, 2016

Apr├Ęs moi, le Tier-3 holocaust

Well, it finally happened: Mozilla's going to end support for 10.6-10.8. Actually, I'm surprised it took this long:

The motivation for this change is that we have continued failures that are specific to these old operating systems and don't have the resources on engineering teams to prioritize these bugs. Especially with the deployment of e10s we're seeing intermittent and permanently [sic] failures on MacOS 10.6 that we are not seeing elsewhere.

This confirms my suspicions that Electrolysis would have been seriously problematic on 10.4 and 10.5 given the permafails Mozilla is seeing on 10.6, even if we got it to compile (it doesn't even do that on Tiger yet). That's why we're forking after 45ESR, which doesn't have E10S enabled by default.

Interestingly, Mozilla's plan is nearly identical to ours: move 10.6-10.8 users onto 45ESR as well so that they'll still get a year-ish of support on that branch in a sort of graceful wind-down. The difference is that we plan to make hacks to the core to support certain post-45 features (making TenFourFox essentially into an OS X Classilla), and of course backport future security updates from 52ESR and so on. I guess if you're running Snow Leopard you can use TenFourFox post-fork with the usual limitations, but I still currently have no plans for an Intel-native version unless someone steps up to maintain it (although I have a 10.6 machine that might benefit, I just don't have the time right now).

The interesting part is that Mozilla still has more users on 10.6 than 10.7 and 10.8 combined, and 10.6 still accounts for something like 13% of all Firefox Mac users. As late as 2014 19% of all Macs ran it. Snow Leopard really is the Windows XP of the Intel Mac.

Mozilla also did a wholesale pruning of most of the other Tier-3 ports, as part of shifting from autoconf:

Following our official move off autoconf, the core build team is faced with having to convert more than 17k lines of shell+m4. A large part of those are to support Tier-3 platforms such as Solaris, HPUX, AIX, etc., with compilers that are not MSVC, GCC or clang (e.g. SunPro, XL C++...). To simplify the conversion, from day one, we will support only the following set of platforms and toolchains:

Platforms:
- Linux
- Android
- Windows
- OSX/Darwin
- multiple flavors of BSD (kFreeBSD, FreeBSD, DragonFly, NetBSD, OpenBSD)

Toolchains:
- MSVC
- GCC/mingw
- clang
- clang-cl

This is a little different than the end of 10.[678] support, however:

This does not mean Firefox/Gecko/etc. will forever drop support for these platforms and toolchains. It simply means we can't assess what parts of the shell+m4 are relevant (as opposed to cruft accumulated over years or decades) and will continue to work (since we don't have automation to verify these configurations).

[...]

The good news is that dropping support now will help make the overall conversion happen sooner, *and*, once the conversion is done, interested parties can come back with working patches that should be easier to write.

My SPARC Ultra-3 laptop (a rebadged Tadpole Viper) runs an ESR build of Firefox just fine in Solaris 10, the last Solaris supported on that platform. Unfortunately I don't have the Sun compiler, or I'd take a whack at building it myself. At least someone out there is still popping out contrib builds for SPARC and i386, but I don't know if they will continue to.

On the other hand, some of those other platforms haven't worked in years. I know of a Firefox 3.6 for AIX, at least on POWER (my PowerPC AIX systems barf on it), but HP/sUX ended support somewhere around 3.5.9 and was always a mess to build (I hated HP's ANSI C-compiler back in the day when I had to administer those systems). I even remember seeing some Digital UNIX/Tru64 stuff deep within xpcom/, but I can't imagine full support persisted much beyond Mozilla 1.8 something. None of the rest of them got much further.

Really, these are all dusty legacies of how portable the old codebase used to be. At one time, Netscape ran on Windows (as early as 3.1), Mac OS (System 7 and up), OS/2, Linux, Digital UNIX (Tru64), SunOS 4, VMS (VAX and Alpha), Solaris, BSD, HP/UX, IRIX, AIX, and probably some other minority ports I don't remember, and all with almost total feature parity. In fact, I myself have personally used every single one of those ports at one time or another. As late as Mozilla 1.7 almost all of those platforms were still working (SunOS 4 and VMS were gone, and Mac OS was replaced with OS X, but the rest were still functioning), but Firefox 2 and 3 gradually winnowed the rest, and by Firefox 4 it was pretty much just Windows, OS X (including us), Linux, the BSDs and Solaris. Soon it'll just be those OSes on x86-64 and ARM.

So, while it may have been hell to maintain, that's a lot of history gone that I personally lived through and good cause or not it still makes me sad to see these last remnants drift away, even on the abandoned platforms. (You can cram it if you're going to post some uninsightful comment like "paying back technological debt" or "no one maintains platforms for free." I'm well aware of the cost of cruft. That doesn't mean it doesn't have historical value, and that historical value should be appreciated, even if doing so is best done in archives rather than current code bases.) It used to be neat to compare how heterogeneous and diverse computing platforms were back in the day, but sadly today's platforms are more alike than they're different, and I think we've lost something there.

It's not really fun anymore.