The sputtering last stand of methodjit

I finished the port of 22 early so that I could enjoy my vacation in June in peace, at least theoretically, and this post is being made in the debug build. (Shout out to vasi: your download progress indicator patch only needed minimal changes for 10.4. I just had to add a stub for NSRectFromCGRect and it seems to work fine.)

OdinMonkey (a la asm.js) landed in 22, Mozilla's hyperoptimizeable subset of JavaScript which appears to have a critical reliance on IonMonkey; the JavaScript test suite can't even complete now without the Python test harness going crazy on those particular tests along with all the other accumulated intentional failures. The loss of regression testing, even though the browser appears to still basically work, should be considered an extremely crippling limitation; in the worst case scenario where we don't make it to 24, I think it might be better to make the more robust 21 the point of dropping to feature parity since I still have a test suite that at least completes in that version. More testing is needed, but in any case, I don't see any reasonable way that gluing together methodjit is sustainable now and 22 will be its last stand. For 24, the only way out is through.

Jan de Mooij at Mozilla did get back to me with what needs to happen for the Baseline compiler, which as previously mentioned is the low-profile version of Ion, and that will be critical for 24 (issue 224). Baseline is a lot simpler to implement, but it does a lot less, of course. The good news is that about 2/3rds of what it requires I'd already done for Ion, leaving only some assorted leftover pieces that I need to write (I also have to get it to at least link completely, what I'm calling "Ion stage 2"; even though not everything needs to be implemented, the stubs need to be present). However, looking at how the compiler works, I'm gloomily predicting benchmark losses of around 25-30% slower. It's still faster than TraceMonkey was, but it's going to be slower than 17. I can't do much in the 22 timeframe other than get it to build completely because Baseline wasn't enabled until 23, so no testing will be possible this cycle.

In that sense, it's probably a good thing that people are only just now noticing that I dropped plugins completely in 19 (yes, it's true, for those new to this blog), so some may stick with 17 for a little longer for the plugins and as a fringe benefit will still have methodjit at near its operational peak. While I'd like to eventually get Ion up on 24 stable, it's going to be entirely dependent on how much backporting I have to do. Such is the nature of life in twilight.

Mozilla shoots us in the stomach

Bang (previously, previously).

So I'm pretty displeased about this, and once the SPARC and MIPS consumers find out that their respective JavaScript methodjit compiler ports no longer work either, they, too, will be displeased. And SeaMonkeyPPC guys, methodjit was sure nice while it lasted, yes? (You'll lose it in the equivalent for Firefox 23.) But if we were beloved by Mountain View, we wouldn't be a tier-3 port, would we?

However, since I'm a physician by training and medical metaphors simply flow from my fingertips, severe traumatic stomach wounds can be survived with surgery. (See how well I pulled that together? My kid sister is a trauma surgeon. You should only panic if I say that Mozilla shot us in the head.) And I'm trying to operate on this patient as quickly as I can between, you know, having an actual day job in a nearly totally dissimilar field.

Some of the JavaScript team at Mozilla are suggesting that we try implementing the Baseline compiler first, which seems a little strange, since Baseline came on board after Ion. The way it's supposed to work is that Baseline builds a really simple unoptimized code stream -- no type inference or other kinds of "expensive" optimization -- and after a certain number of runs, then big bad Ion comes on and emits superior code, and that runs instead. In fairness, this seems to work as a one-two punch better than methodjit, so I'm not unhappy about the end result, just the fact that Mozilla has really boxed me in here. Implementing Baseline first does have the advantage that there is some common code, I can pull Ion up in stages, and it would be better than the interpreter (but that's like saying rotten fish for dinner is better than nothing for dinner). And Baseline is easier on the processor to compile code, similar to the way TraceMonkey worked, so really slow systems like our G3 users will notice that latency is less even if the code is not as well optimized. But our benchmarks may take it on the chin, and like Mozilla would agree, I don't like shipping regressions.

I'm still waiting for my JS contacts to explain in more detail what's needed to bring Baseline up. The problem is that there's no fully functional Baseline in 22, and there is no point in building 23 -- we want 24 because we want to move to the next ESR at (almost) any cost. So I'm going to hold our stable users on 17 as long as possible. 22 is going to come out, the last version where methodjit was relatively unmolested, and I will start on that port this weekend now that beta 1 is available so I can still take some time off (which I will spend dreaming of throwing darts at Mozilla's corporate office). That, and putative 17.0.7, will still come out on time as usual.

Currently Ion is to the point where it builds, but does not link -- I'm still missing some stuff which I am implementing piece by piece (not yet to phase 2). When 24 hits Aurora, then I'm going to try to get Baseline running first assuming Mozilla isn't blowing smoke up my tailpipe and run numbers. I figure with the work I've already done, getting Baseline to pass tests is going to be at least a full cycle, so there will be no Aurora for you to play with unless there is a major breakthrough (putative 17.0.8 will be released, though, and if there is a major security issue I will put out a bridge 22 release). If I'm lucky, there will be a 24 beta for you in the beta timeframe prior to 24.0 final. Play with it, see if the expected regression is still worth shipping, and Chris T and our localizers can start work with that. If people judge it acceptable, we issue 24.0 on the unstable branch and 17.0.9 on the stable branch to have one more shakedown for bugs, and move stable branch over with 24.0.1/17.0.10, and 17 support ends (I hope Tenfourbird is able to jump to 24 with us -- I would love to see that project continue). If the benchmark regression is not acceptable, we'll just have to hope there's enough time left on 17 to get Ion ready.

This sucks.

17.0.6 available

17.0.6 is available, word to your moms' on Sunday, on the Downloads tab (release notes). This also includes the issue 217 fix that appeared in 21.0 on Wednesday. Assuming no problems, this will be final on Monday.

21.0 released

First, let me issue a capsule music review by saying that The Beach Boys' Sunflower is their best work since Pet Sounds, and anyone who knows my taste in music knows the high regard in which I hold the latter. Brian Wilson's muse guides me as I write the following. Thus concludes my channelling of Lester Bangs, rest in peace.

21 is released (release notes). This is not really a groundbreaking milestone, but it is a noticeably quicker one; we're finally getting on the other end of the JM+TI curve after we took a bad hit around 19-20. In fact, V8 is about 10% faster on the G5, mostly due to improvements to regular expressions, although I'm still unhappy with V8-Crypto's performance and I'd like to unregress that if we end up stuck on JaegerMonkey. I also got canvas compositing working with the 10.4 CoreGraphics backend, though I really need some reftests to verify if it's actually doing it correctly (but to my eye and some basic examples, it appears to be). 21 also includes a fix for a crash in the CoreGraphics backend (issue 217) which will also be fixed in 17.0.6, due out this weekend.

21 also includes the Firefox Health Report, which because of our branding work comes out as the "TenFourFox Health Report." I'm not sure what to do with this yet. It's disabled because the stuff currently gets sent to Mozilla, not me, and I imagine they don't really care. The time and performance standards are also based on computers considerably newer than our cheerfully geriatric workstations, so I don't even consider them relevant. I'll probably completely strike it from the UI for 24. Play with it if you like; I don't really see this as worth building infrastructure for, since the necessary knobs to twiddle it are mostly out of my control.

Under the surface, JavaScript is still kind of a mess, even though it works at least right now. We're getting more and more divergent from Mozilla, so we pass tests we shouldn't (mostly due to stack bailouts, which interestingly don't make us crash anymore because we have so much stack and we allow big stack allocations -- people who remember early problems with stack crashes in the TraceMonkey days will probably snicker at this point), and our methodjit is hacked to avoid certain performance regressions introduced during the Ion-Jaeger integration work, which makes us slightly out of step with an obscure feature.

The IonMonkey work still doesn't build yet, though the assembler and the macroassembler do parse; the lowering and move emitter code need to be reworked now that the former two pieces are done using different assumptions, and then stubs need to be written to let it link. At that point it should be able to crash successfully, sigh. Mozilla has not made any promises about keeping JaegerMonkey in 24, and those of you using Firefox on Linux PowerPC may wish to keep an eye on the interpreter changes because it's being drastically simplified (and this is likely to make using the interpreter even less performant, though given that V8 doesn't even have one you should be consider yourselves fortunate there's an interpreter at all). If the worst case happens and Mozilla yanks JM before the 24ESR switchover, I might issue interim 17 and 22 releases until ionjit gets mounted, but I really don't want to be in that position. Alas, Ben's PowerBook is out of commission, so he is unable to contribute. You can help! Voice your love, affection and ability to program in issue 178.

Speaking of 22, I'm not terribly worried about that port, since JM-solo builds were still supported in this release; there might be some build system changes to work around, but the merge work should not be major, he says while rapping his knuckles on a convenient plank of wood. This port will start early because I'm actually due to take a vacation (ha! fat chance) in June for the first time in nearly two years, and I want to make sure the bulk of the porting work is done in advance. But don't worry, the 22 and 17.0.7 builds will still be out on time. Don't worry, baby. Everything will turn out all right. Smile.

A scrollbar too far (or, why Apple can't kill a Snow Leopard)

One of my perpetual sources of anxiety is whether when Mozilla will drop support for 10.6 Snow Leopard. We've done pretty well keeping 10.4 and 10.5 compatible code in our widget library, and it looks like the code landing for Australis (the new "look" for Firefox) will be compatible with only moderate tweaking at most so far. Because Snow Leopard is essentially a tuned 10.5 and has relatively few UI-related changes, our now fairly streamlined widget library code functions quite well, considering, on all three operating systems (essentially forward-porting the 10.4-specific stuff from 3.6 and the 10.5-specific stuff from 16, for those weirdos in the audience running TenFourFox on Rosetta in 10.6). There haven't been substantial changes to our widget patches since 19, and even then only for the compiler update.

But then of course there's 10.7 Lion, Steve Jobs' Operation Market Garden, where scrolling was screwed and Rosetta was removed and Java was jettisoned (wait, that's a good thing) and scrollbars were iOSified and Save As was deprecated and you can't tell if an application has quit or not. And like 10.6 left behind an entire generation of Macs -- every Power Mac ever made -- 10.7 left behind every 32-bit Intel Mac, and 10.8 won't even run on those 64-bit Intel Macs that can't boot a 64-bit kernel, including my 2007 Core 2 Duo Mac mini which I use for taxes and Android stuff. 10.8 does smoothe out some of the rough edges of 10.7 but doubles down on them in other respects, and why bother downupgrading to 10.7 if I can't upgrade to 10.8 as quickly as possible thereafter? I could boot 10.7 on my mini, but then I lose all the features I need from 10.6, I can't upgrade to 10.8, and I have no apps that demand either one. So 10.6 stays.

In that context, then, I suppose it's not surprising that Mozilla (and probably Chrome as well) has a problem: 43% of their users are on 10.6, an OS that Apple is supporting pretty much in name only right now and will almost certainly cease to after 10.9 emerges (expect word on that from the incredibly oversubscribed WWDC). And why is this? Well, look at the other two numbers: while 10.8 is growing, 10.7 is at 30% and dropping. I can't be the only one who made the calculation that it's better to stay on 10.6 if you can't go to 10.8, and there are still lots of 10.6-only machines in use that really aren't that old. Since Apple isn't maintaining Safari anymore for 10.6, it's alternative browsers ahoy.

Unlike previously premature attempts to kill off platform support on Mozilla's part (ahem), there's a big operations problem brewing here. Apple is obsoleting hardware faster and faster, and Mozilla can't buy more 10.6 or 10.7-capable systems because Apple won't make them. (Used hardware is not generally acceptable for the types of testing they need to do; their Mac mini build farms work very hard.) They might be able to build on a later version of OS X (but not for much longer?), but they wouldn't be able to test how well it works on earlier ones. So they're going to repurpose the 10.7 test systems for 10.6, since it continues to be the majority of Firefox's OS X user base, and make 10.6 and 10.8 the major test platforms until Apple finally has a developer beta of 10.9.

All this is good news for us, of course. On the dark day when Mozilla makes 10.7 the supported minimum, all supported Macs will run a 64-bit build, have hardware acceleration, and use the new UI and related libraries -- none of which is true for us (we still use some Carbon code, limiting us to 32-bit even on G5, we don't have hardware acceleration, and our scrollbars are actually visible and scroll in the correct direction). At minimum we would have so many changes to pull forward that we would almost certainly have to split off separate widget and theme subsystems, and merging work would increase significantly if we could get it to function at all. But with almost half their users on 10.6, even considering that the number is slowly dropping, I'm now not anticipating this will occur until at least the next ESR and the chances of a working TenFourFox 31 have just increased big time.

IonMonkey PowerPC phase 1 complete

Tonight after several weeks of feverish work I finished the basis for IonMonkey PowerPC, the next-generation JavaScript JIT we will be implementing, including bridging our previous assembler (and all Ben's hard work on optimizing branches) to IonMonkey and then writing up the new macroassembler. IonMonkey is all stack, all the time, so there are quite a few differences in implementation (though to the Linux readers in the audience, this may make our IonMonkey backend easier to port to Linux PPC than our current JaegerMonkey/methodjit backend because I can no longer make certain assumptions about how the stack is organized). The old macroassembler still remains as an integral part of YARR and is still used to compile regular expressions.

Phase 2 will be getting it to compile, phase 3 will be getting it to do simple operations, and phase 4 will be getting it to pass the JIT test suite (which if it does means there is an excellent chance it will "just work" in the browser). Please note I am still unconvinced about how well it will perform, though the front end for IonMonkey can optimize code a lot better than JaegerMonkey could, and because it does everything on the stack I hope our 1GB allocation is enough because I don't think we can get any more out of it in 32-bit mode.

Mozilla has at least not laughed uproariously and said "no" with a bone-crushing sound to the plea to keep JM+TI (JaegerMonkey + type inference) in ESR24, even if only as a compile-time option. There is no SPARC ionjit (nor does it look like there will be one), nor MIPS, so those architectures will also lose JIT support when JaegerMonkey is removed, and since it appears that JM+TI's continued existence does not immediately impair work on certain pieces of the ionjit baseline compiler there is less push to remove it right away. This was not the case per se with tracejit back in the ESR10 days where tracejit actively impaired work on type inference. I really don't want to have to unload ionjit on the user base right away; I'd rather have lots of cycles to get it right, and keeping 24 on JM+TI means we have one more full stable release lifecycle on a very dependable engine that's bought us a lot of mileage and plenty more time to get Ion to that same level.

FirefoxOS dev phones sell out

By which I mean they're out of phones for sale on the very first day, and I blame you lot for why I can't buy a Peak Developer Edition. The specs are modest, but the price is eminently reasonable. Last night at 10:30pm the store was down for maintenance, I get up at 5:45am today to get one, and bam.

I'm a longtime Android user, driven to the Nexus One when Apple dropped 10.4 support within the iOS 4 timeframe, and while I don't love it I certainly don't hate it. My Galaxy Nexus is a solid device. Still, when I got to play with a Palm pre 2 (thanks, Ed!) for a period of time it was a delightful machine, and I'm hoping FirefoxOS recaptures that ease of development and feeling of openness that webOS had. Bugs are to be expected in early dev releases like this, and a friend of mine who put a nightly on a castoff phone a couple months back reported it didn't even do cut and paste (!), but I expect it has now developed to the point where you can dogfood it and I certainly hope Mozilla is. If FirefoxOS launches well, it would do much to preserve their relevance and a freer alternative mobile platform in a post-PC era.