Further work on 31

I am pleased to announce that the bugs mentioned in the last blog entry are all stamped out of 31, including editor key combinations now working again, webcam video+audio working again, the endian bug in the JavaScript VM found and, of course, a new compatible QTE. This blog entry is being typed in it. As far as I am concerned, absent a new major bug discovered during your testing, 31 is now in a shippable state.

There are two regressions I am tracking and both are big question marks. Chris mentioned a WebM performance regression; I can't reproduce this on the quad G5, but the G5 never had a problem with WebM even before the AltiVec patches went in, and my usual iMac G4 tester never ran WebM without a problem. I've turned the loop and bilinear filters down as low as possible, with fortunately minimal visual change on most YouTube videos, and made sure that video frames are being passed directly to the compositor. I hope that helps; I don't really have anything else to offer there. libvpx changed dramatically in 29 and it might be that it's just too heavy now for the G4s that used to run it acceptably.

I'm pretty sure, by the way, that OMTC (off-main thread compositing) is not to blame for this, which was also enabled in 29. OMTC puts the compositor on a separate asynchronous thread so that compositing graphics layers together doesn't block the main business of the browser. Multithreading is not great on Power Macs running OS X and is worse in 10.4, so OMTC makes some sites scroll better and some sites scroll worse, but OMTC is a prerequisite for future features like Async Pan-Zoom which finally lets you move around on the page while the browser is working on it and Mozilla is openly warning they won't support synchronous on-main thread compositing for much longer. OMTC is the future and fortunately it works on Tiger, so we're moving forward with it.

The second regression is JavaScript. While generational garbage collection (GGC) improves browser memory usage even more, it comes at a price; the changes for tracking these new short-lived objects add further overhead to the JIT and even with some additional fiddling we still take a hit of about 10% relative to 24. The G5 takes it on the chin even more, about 15%. This is still substantially better than the naked interpreter, but the interpreter is now so deoptimized that this isn't saying much anymore.

The biggest decline was due to us failing to run a particular type of analysis which is built into Ion, but we don't run Ion, so this didn't run and PPCBC generated terrible code as a result. I hacked this back together and it now works mostly as it did before, but this makes the definitive solution clear: we need IonMonkey working on PowerPC or we'll end up incessantly running into these kinds of edge cases in future versions. Plus, more to the point, Baseline Compiler was never intended as a definitive JIT solution despite the fact we're using it like one. My previous efforts with Ion ran aground in the bailout code and I think the main problem is related to Ben's sophisticated but complex branch optimization we used in JaegerMonkey (the previous JIT), so that needs to be decoupled from the low-level assembler which I'm working on over the weekend and then Ion dragged back up to a testable state. If I can make progress on that, I am considering delaying 31 until Ion passes tests (no asm.js; I continue to believe that it's irrelevant on big-endian because of all the little-endian code compiled for it) like we did with 24. If I can't, we'll just do the best we can.

For now the current plan is a 31 beta shortly after 24.6.0 which will still use PPCBC unless I have a major breakthrough; after that we'll see. 24.6.0 is still on schedule for an on-time release the second week of June.

31: close to beta (plus: new QTE and Mac|Life goes Low End)

Good news: there's going to be another stable release; 31 builds, links, gets off the ground and generally works. Yay! 24 will go into maintenance mode until 31 is ready for prime time, but the chance is better than even money that we'll have an on-time launch.

Between 29 and 31, Mozilla reworked the basic layers canvas code and this fixed the display problem with Google Maps and a lot of other sites. Plus, there don't appear to be any new regressions with the graphics code switching to Moz2D, and our "blue thumbnails" patch sticks fine in 31, so that's a lot of potential problems avoided and repaired.

The four major bugs we have left are all holdovers from 29, except this one: Google Maps does display correctly now, but it crashes with a null pointer assertion within the JavaScript virtual machine even with the JIT off. This looks like an endian problem introduced with bug 912456 or a related patch; simply wallpapering the assertion to make the browser continue doesn't cause any crashes, but Google Maps won't download new map content, so that's not very helpful. I consider this a showstopper. However, I also have a good idea where the problem is.

The second major bug is that the current version of the QuickTime Enabler does indeed hang up in TenFourFox 29+, as reported by a couple folks earlier. This is a problem with the old Add-on SDK jetpack version in v116, confirmed in that the MacTubes Enabler, which has a later jetpack, doesn't do this. Both of them will need a jetpack update for 31, but I updated the QTE jetpack and added a couple minor bug fixes and released that as QTE v120. You can get it from the QuickTime Enabler wiki page. It will work with 24 and 31 both, and then the QTE and MTE will be refreshed again after 31 is released. So that should be solved.

The two remaining issues are also 29 holdovers and while I consider them important to fix, I don't consider them showstoppers. It does look like editing keys have regressed per your reports, but not all keys have, so I'm not sure what's going on (menu keys and productivity shortcuts, for example, all function). This looks like our bug, but while I intend to fix it I don't consider this a shipping blocker for 31.0. Also, we still have the problem where webcam audio does not work (and neither does video-with-audio). However, our patch to force webcam video and video-without-audio does work, so the webcam can still be used for snapshots as its primary use case, and this is also shippable even if it's suboptimal.

31 also uses Mozilla's new generational garbage collector (GGC), which is even more aggressive about memory than the exact rooting GC in 29. The settings 29 use that we experimentally determined earlier in this blog are tuned for the older conservative garbage collector and may be counterproductive with GGC. I will likely change the time slice and some other parameters for 31 beta, so if you are using these custom GC settings please revert them before upgrading or the browser may not perform well.

Other than that, the browser appears to work fine and I think we're in good shape for another successful stable branch. Once I have the Google Maps problem fixed, I'll grab the beta source when 31 migrates from aurora and we should have a full-spectrum 31 beta release available a few days after the 24.6.0 update. Speaking of, 24.6.0 will be a security update only; there are no bugs on our side that will land, and I am not likely to do further feature work on 24 as the switchover to 31ESR approaches.

By the way, if you've got a newsstand nearby, you might want to pick up the current (June 2014) issue of Mac|Life and their article on rejuvenating your old Mac:

Even though it's a relatively basic article and their tips won't be big news to most of this blog's denizens, it's nice to see classic Macs still getting coverage in major Mac publications, and they even have a little section on Linux and alternative operating systems. Thanks to @thedoctor on App.net for spotting their shouts-out to TenFourFox and Classilla -- that's how you know you've arrived.

Not MIA, just BUSY

I'm still fighting with the server firmware to get uppsala's second core (which I paid good money for) turned back on, but in the meantime 31 is coming along; the patches are down and JavaScript at least builds and passes tests with the new generational garbage collector. However, changes made to accommodate the MIPS port and some other tweaks have degraded our JIT performance, so I need to pump that back up before continuing. Then we'll continue with the rest of the browser.

At the same time I'm looking ahead to see how feasible the ascent to 38ESR will be. Even money says that Mozilla and Google drop 10.6 support (and quite possibly 10.7 support) sometime in 2015, but if we can stall them to drop it for 38 but not remove any of the code like they did for 16-17 when 10.5 support was removed then we can probably make it. At that point we would exit the 38ESR support cycle in late 2016 when the youngest Power Mac will be turning 10 years old. If we have to drop source parity then, it would certainly not be in shame.

For JavaScript in Fx32 Mozilla has implemented irregexp, V8's regular expression engine, and dumped WebKit's YARR which was dragging down benchmarks. I was initially very worried we would need to implement yet a third JIT ourselves, but to my great delight it looks like Mozilla's adaptation uses the existing macroassemblers, so we just need to make sure there are no endian problems in it.

And more good news, this time for Linux/ppc users; at last there's apparently a solution to the infamous bug 961488. If it works, we'll see if it can be uplifted so you won't be out of commission for too long.

And we're back

Thanks for your patience. Other than a missing core which the vendor should be able to activate remotely, uppsala is back in service and everything should be pretty much back to normal. I am relieved to note that despite the need to wipe the RAID cache, no data appears to have been lost. One wonders when Apple will finally go to a "big boy" filesystem like AIX's JFS/JFS2, which has never let me down in two decades, as opposed to HFS+ which can poop its pants on a bad day.

System downtime update

The news has gone from bad to worse; it looks like uppsala, the big POWER6 that powers Floodgap, actually blew its mainboard and unfortunately the RAID array might have gone with it. A new mainboard should arrive today or tomorrow and we'll try to assess the disk damage when I get it installed (total repair bill so far $1700). If we're lucky, it's just the write cache and there weren't critical sectors in it, and the filesystems will be sane or at least somewhat recoverable. If we're not, we're starting from scratch and I'll have to rebuild the server; most of the data is backed up, but it's going to take some time. Either way, stockholm's going to be staying on active duty for at least a couple days more. Thanks for your patience.

Floodgap service note: emergency service reduction until further notice

Instead of working more on TenFourFox 31 aurora, I spent the evening putting stockholm, my trusty Apple Network Server 500, back in service; uppsala, the big POWER6, threw a rod sometime this afternoon and won't pass power-on diagnostics. Fortunately it had done its normal automated backups earlier, so there will be minimal data loss and the hard disks should still be okay. Let's hope it's something easy and cheap(er) to replace, like the RAID controllers or the accessory Ethernet, instead of replacing the entire system backplane which is basically an insanely expensive motherboard swap.

However, the ANS is just a little 604e with only 512MB of RAM and is not up to the task of serving everything the POWER6 does. I've restored static images of the TenFourFox, Classilla, HTTPi, Texapp and TTYtter sites and some utility areas along with the update XML files so that checking versions will work, and ordinary users won't notice much difference except that the site will be a little slower than usual. However, there are no gopher services and most of the rest of the website is down; really stockholm is just here to serve critical pages and collect E-mail until repairs are complete. That said, I'm glad I kept it configured as a server for just this sort of situation, and it's nice to have the old beast slip right back in to keep things running.

Hopefully my rep can overnight me the parts and we can get this fixed by Tuesday or Wednesday of next week.