24.0 might actually work out after all

I am very relieved to announce that TenFourFox 24.0 builds and links, and after a couple of tweaks does get through the low-level initialization into basic XUL and an event loop (I haven't tested it further yet since I want to back up my profile and a few other things first). A lot of hacking was required for 24's initial underlying work for Australis, including adding HIToolbox equivalents for more dependencies on CoreUI (not in 10.4) and I'm a little worried at how incredibly divergent our already weird widget code is becoming. I also had to do some surgery on the guts of the profiler, which doesn't work right in 10.4 but is becoming an increasingly large part of Mozilla's telemetry initiative and has to be worked around just to get the app to even start.

Speaking of Australis, Mozilla's planned new super-sexy interface for Firefox, its continued tardiness has become the big sniggering joke of the web browser blog circuit. It was supposed to land in 25, but that obviously slipped (I'm using 25 beta at work, and it's "regular Firefox"); it doesn't appear to have made it into 26 aurora either, so maybe we'll finally see it in Firefox 27. Maybe. Mozilla isn't really saying and I'm not sure where the hold-up is since most of the performance regressions we care about have been mitigated. Australis is one of those must-reach goals, like having some sort of JIT; whenever it lands, if we can't move forward, that's it for the port. I'm hoping it lands in 27, and then we can do some feasibility studies and roll back to 24 if needed.

In light of 24.0 actually building and linking, regular readers will wonder where the phase 4 announcement is (passes all tests) for PPCBC, our new baseline JavaScript JIT implementation after Mozilla removed methodjit in Firefox 23. Well, there isn't one -- it's still failing three of them. One is stubbed out with a tweak to the compiler itself since I suspect the underlying problem is actually a Mozilla bug I can't track down yet, and the second is that endian issue I've previously mentioned with asm.js which is pointless to fix since those sorts of scripts are usually hopelessly little-endian dependent anyway (methodjit doesn't pass it either). The third one does concern me a bit; it's an issue where we can overflow our JIT branching with really unlikely chains of try/catch blocks. It's not a security issue and the code necessary is terribly pathological, but it is a reliable crash, and I have to think of a way to repair it that won't drag the browser down. I don't think we need it to launch, though.

Once the last few JavaScript issues are shaken down and the widget code is verified to be working again, the last thing is to look at getting jemalloc back if the memory issues we discovered in the 22.1 test release are still present. It really does help (when it works) on multi-CPU systems on 10.4, even though you, my testing audience, found little improvement in single-CPU systems or on 10.5. The drawbacks are at least well understood, but if the memory leak still manifests we'll still have to rely on the default and slower built-in OS X allocator.

This will be tight for a beta, but at least we have two full cycles to get it right since 24 final will not be out until the 24.0.2/26 timeframe, six weeks after our final 17.0.10 release. Watch this space for more.

Because of all the widget work Australis is containing, I'm considering pulling the trigger on not allowing TenFourFox to link against the 10.5 SDK anymore. This has not been well-tested since at least 19 or earlier and it is quite possible it no longer works (Tobias' AuroraFox for 10.5 was actually a separate set of patches with Leopard-specific code, not what TenFourFox uses). You would still be able to build and run TenFourFox on 10.5, of course; you would just have to link it against the 10.4 SDK, which would become the only supported target. This would allow me to simplify a great deal of the code and improve maintainability when we split our widget and skin code off the mainline. If this is going to desperately break someone, speak now; my idea is to make 24-stable the last release series where linking against the 10.5 SDK remains even only theoretically supported.

The great "I told you so" on plugins

*ahem* I told you so.

Google is not banning every plugin -- Flash still operates in Chrome -- but it is banning the NPAPI plugin family to which nearly every extant plugin belongs and entirely switching to its Pepper plugin API protocol, Chrome's closed source private implementation of Flash, the Chrome PDF viewer, and Google Native Client being the only known PPAPI plugins. There have never been PPAPI plugins for PowerPC OS X, and no Power Mac browser has ever supported the protocol.

A little history is in order. NPAPI stands for the Netscape Plugin Application Programming Interface, and dates all the way back to Netscape Navigator 2.0 when Adobe reverse-engineered Netscape Navigator to allow PDFs to display natively in the browser. This was "Allan's Hack," named after Adobe developer Allan Padgett, who convinced Netscape CEO Jim Clark to put resources into an API to generalize the process. It was gradually expanded in scope with the modern interface descending from NPRuntime circa 2004. Virtually every browser plugin in existence today is based on the NPAPI spec.

Like our plugins page says, plugins bring lots of concerns with them owing to NPAPI's permissive design: they can destabilize the browser, spy on it, or do anything that any other application can with the credentials of the browser user. Plus, being blobs of proprietary code we don't control, we can't update them or easily patch them. Now that Mozilla has removed the QuickDraw and Carbon code on which every Power Mac-compatible NPAPI plugin depends, we don't support them at all, and after the release of 17.0.10 ends support for TenFourFox 17, no supported release of TenFourFox will have any plugin support in any fashion.

Google, because Google wants to redefine the web and eat it, tried to issue a competing "standard" in 2009 called the Pepper Plugin API (PPAPI). PPAPI has some advantages such as proper sandboxing and arguable performance improvements, but calling it a standard or even just a specification is somewhat disingenuous of Google because PPAPI isn't really a true spec like NPAPI was; it's mostly exposed hooks in Chromium that a plugin library can link against, which is why Mozilla refused to support it in Firefox: it would either require importing large portions of WebKit Blink and/or Chromium (Google even uses Chrome and Pepper synonymously), or putting lots of hours into creating an API that looks just like it, and then dealing with the moving target of additional functionality being rapidly added to the core using a description that was obsolete the day it was published. Google engineers have even admitted off-the-record that the Flash Pepper plugin in Chrome depends on APIs that aren't even documented.

Either way, plugins as we know them are on their way out. Mozilla doesn't like them because of their security issues and their provenance as opaque and possibly untrustworthy binary blobs, and Google doesn't like them because they're not under Google's thumb. In this circumstance, think of TenFourFox as merely being ahead of the curve.

22.0.2 available, damn it (plus: burning Dreamcast discs on the Mac? yes, OS 9 can)

I am stuck on the last seven failing test cases for PPCBC, so I'm biting the bullet and issuing one more interim release of 22 with a couple bug fixes and security patched up to parity with 24 (a couple unlikely situations are excepted, because they would be almost impossible to exploit the way we are constructed); downloads and release notes in the usual places.

The failing PPCBC test cases seem to revolve around issues with the JavaScript debugger and telescoping JITted code within JITted code through the standard OS X ABI (jumping code fragment to code fragment within Ion's internal ABI works fine). Unfortunately, these must be solved; the debugger will crash if they aren't, which is obviously unacceptable. And, of course, there's still that failing asm.js test with the endian issue, but it doesn't cause a crash and methodjit never passed it either, so I'm not counting it right now. I don't know how long it will take to solve.

In the and-now-for-something-completely-different-dept., I am a big fan of the Sega Dreamcast, their last console and in fact the last console I ever bought new. (Subsequent to this, I was given a slimline PlayStation 2 by a friend, and the only usage of that console is in fact whenever said friend is over to visit. Perhaps said friend would like it back.) The Dreamcast uses a perverse format for booting discs that requires a multi-session XA CD-ROM with an audio track and then a specially written data track using an embedded bootloader at an offset. To make this easy, most ready-to-go Dreamcast images (we don't condone piracy here; I'm talking demos and utilities) are in DiscJuggler CDI format, but this particular program has never existed for Macs (Liquid CD will attempt to burn them, but they don't work) and the existing Mac CDI extractors are of questionable functionality. Making Dreamcast CD-ROMs on a Mac has been an exercise in frustration since the beginning for virtually everyone who's tried it and now most Mac Dreamcast enthusiasts just Boot Camp into Windows and do it there. That's not an option for Power Macs, of course. I've personally tried for years and never been successful.

To this day, no disc I have ever burned from scratch on my G5 has booted my Dreamcast. That is technically still the case, but I am at last able to make bootable discs with the help of, incredibly, good old Mac OS 9. I discovered the trick when I stumbled upon a Mac-native (for OS 9) set of sessions for the infamous Utopia boot disc, which besides its more nefarious purposes serves to allow you to play legal games from Japan in, say, a US Dreamcast. It was very specific that Toast 4.2 had to be used -- my MDD normally uses Toast 5 in OS 9, but I still had 4.2 on CD from when I used it with the SCSI burner on my Power Mac 7300. I dug out 4.2 and installed it. Unsurprisingly, given its age, it refused to use the DVD burner in the MDD, but it would use the combo drive I'd added to the second optical bay. I put on the provided audio track for the first disc session, put on the provided ISO for the second session, and put it in the Dreamcast. It worked!

So I tried to replicate this with Toast 5 on Mac OS 9. Toast 5 couldn't make a bootable disc with the same files.

I then tried to replicate this with Toast 6 Titanium on the G5 and the same files. Toast 6 Titanium couldn't do it either. One page suggested burning the track as an Enhanced Audio CD instead of a regular Audio CD. This made some sense since these are technically MIL-CD format discs, but that didn't make a difference (and that's not what Toast 4.2 appears to be doing anyway).

I even tried to replicate this with cdrecord burning a blank audio track for the first session and using the same ISO. cdrecord couldn't do it either.

In fact, to get any Dreamcast disk bootable requires me to go into Toast 4.2 on the OS 9 MDD and use it specifically to burn the first audio track (at 1x) as a session. Once that's done, I can put it in the G5, regenerate the ISO for the data track at that offset (if you're using mkisofs from MacPorts, it's -C 0,11702 for the standard four-second audio silence track), add the IP.BIN loader to replace the first 32768 bytes of the ISO and then cdrecord that modified ISO to disc. And then it all works.

The key appears to be something weird or possibly "wrong" that Toast 4.2 does when making that critical first session. It's not something about OS 9, because Toast 5 doesn't act the same, even with exactly the same settings. It's not something about using a DVD burner or cdrecord or even mkisofs, because the G5 burns the second session just fine. Whatever it's doing, I can now boot Dreamcast Linux. I'm going to look at some of those CDIs again next, though a lot of them cram extra capacity onto the disc and I don't know of a PowerPC tool that supports overburning. I might see if I can get these to work with the DreamShell SD card reader I just got. But, after literally half a spool of CD-ROM coasters in my trash can, I am finally able to make Dreamcast discs in some form at least and start hacking into one of my favourite game consoles with this surprising new use for Mac OS 9. Whatever it's doing behind the scenes, it's making me very happy. :)

As a matter of fact, I did have a V8

Starting program: /Volumes/BruceDeuce/src/mozilla-24a/obj-ff-dbg/dist/bin/js --baseline-eager --no-ion -f js/src/v8/run.js
Reading symbols for shared libraries .....+++ done

Richards: 192
DeltaBlue: 187
Crypto: 330
RayTrace: 297
EarleyBoyer: 270
RegExp: 230
Splay: 717
NavierStokes: 430
----
Score (version 7): 301

BaselineCompiler PowerPC (which I've christened "PPCBC") now passes V8 with flying colours (it also passes SunSpider). It still does not pass all the tests, but it's only down to a handful and I have a reasonably good idea where some of the bugs are. If I can get my skates on, I might have it passing all tests by next week -- which could mean a 24.0 beta instead of a 22.0.2 interim release. Let's see how far I can get.

The sharp-eyed amongst you will have noticed the numbers, although still much faster than the raw interpreter, are much lower than methodjit's were. Do keep in mind this is an unoptimized debug build running on my quad G5, and the PowerPC 970 suffers badly when its particular idiosyncrasies about instruction scheduling are not indulged in addition to all the debug-friendly but inefficient code in a debugging build. Also, BaselineCompiler's inline caches are very generalized and not well-suited to the PowerPC instruction set. I'm going to be almost completely rewriting those when all the tests pass, which will really earn the "PPCBC" name.

That said, though, I expect PPCBC's numbers only to improve about 3-5x even with a properly optimized build, correct processor specialization and better inline cache codegen. That puts it firmly in old TraceMonkey JIT's range: a lightweight compiler which means its code can come online quicker and run for longer, but less well optimized and not as strong a performer on benchmarks. To fix that will require getting full IonMonkey and that's not on the table yet.

17.0.9 available

17.0.9 is now available from the Downloads tab (release notes). Mozilla has started getting ESR builds out earlier in this week, which thus enables me to get ESR builds earlier to you, assuming they are not pulled or a chemspill occurs. This release fixes issue 239, a patch for an endian problem in methodjit submitted to Mozilla by our friend at Tenfourbird; Mozilla rejected it, though purely for understandable administrative reasons due to the short life left in ESR17, so we're taking it ourselves. By happy coincidence this turns out to also repair long-standing issue 200, a problem with Yandex Mail (and probably other sites relying on JavaScript stricteq). I note by way of self-defense, however, that the bug is actually in Mozilla's code and the testsuite doesn't catch it.

Speaking of JavaScript, BaselineCompiler PowerPC now compiles and successfully passes roughly 3/4ths of the tests, but there are still some significant failures to be repaired. The current roadmap to TenFourFox.next looks like 22.0.2 (sometime next week when the security advisories become available for comparison), followed by 24.0.1 beta (accompanied by 17.0.10, the final 17 release), followed by 24.0.2 release and then an immediate scramble to get 26 off the ground. Obviously if I get a 24 beta working early, this whole time frame can be accelerated, but I am confident we will have 24 ready just in time for the end of 17. There will be one beta cycle to allow our localizers to catch up -- watch issue 42. We'll be loading 24.0.2 final onto our new SourceForge site since the end of downloads on Google Code is just a short four months away, and, you know, phooey on them. Their loss.

What the NSA revelations say about our online safety

This blog is many things; one of them is apolitical. I have assiduously avoided direct commentary on the Edward Snowden saga here because I think we have a plurality of views on the topic and frankly I don't want to derail our main and most important focus, which is to continue the maintenance of a functional, useful and above all secure browser package for PowerPC OS X.

The latest revelation from the Guardian, however, directly touches that last attribute: the National Security Agency (USA NSA) and Government Communications Headquarters (UK GCHQ) are not only gathering encrypted traffic from the Internet, they are able to decrypt large portions of it either through brute force or (and this is the most worrisome part) deliberate weaknesses clandestinely introduced into encryption algorithms, i.e., back door attacks.

Some of you may say, well, planners gotta plan, soldiers gotta fight and codebreakers gotta break codes. But the possibility of back doors in common encryption methods is terrifying because who says they haven't yet been detected by criminals, or worse? Even if the NSA or GCHQ have the absolute best of intentions and are working entirely within their mandate -- discussions we're not going to have on this blog -- the best weapon of anyone looking to loot your bank account is to grab your authentication information over the connection you previously believed was ironclad. As quoted in the article, "[an] encrypted communications system with a lawful interception back door is far more likely to result in the catastrophic loss of communications confidentiality than a system that never has access to the unencrypted communications of its users."

Mozilla's leadership in security is clear; both Firefox and Chrome use NSS, which originated with Mozilla, for encryption and certificate management. I am confident that we'll see Firefox and thus TenFourFox evolve more robust encryption methods out of reach of present-day brute force computation and beyond the taint of the NSA or NIST. I am concerned about Classilla; evolving its encryption to a later version of NSS may be beyond the capabilities of CodeWarrior or Mozilla 1.3.1, and I don't believe it serves the community to continue evolving a product with a security issue I know I can't solve with the tools available. This is a situation we need to watch very carefully.