TenFourFox Development

10.0.3 RC (aka: Kaiser lies again)

2012-03-07T19:42:00.002-08:00

Okay, I lied again. 10.0.3 RC is out (read the release notes), with issue 133 for better WebM on multicore Macs, and fixing issues 132 (black images cause graphic weirdness) and 134 (Collusion extension and other square-root-dependent code on G3/G4). Please verify all is good for you, and then stay tuned for 11 in a couple days pending Mozilla signing off on the final release. 11 will not have issue 134 fixed in it, on purpose for testing, so expect that.

G3
G4/7400
G4/7450
G5

10.0.3 and 11 beta

2012-03-05T18:08:00.003-08:00

Mozilla has the ESR branch marked with DONTBUILD for some reason, so the ESR RC for 10.0.3 will not be out until this weekend along with a temporary patch for issue 134, which appears to be a bug in our software square root. In the interest of stability and not duplicating effort, we'll leave it off on the stable builds and try to get it fixed in unstable.

Speaking of unstable, this post is coming from TenFourFox 11 ("but ours goes to eleven"). I lied, I decided to do a beta for it after all. You'll get that soon.

10.0.3pre available

2012-02-24T16:43:00.002-08:00

10.0.3pre is available. This fixes the issue with black images (in a better way than previously planned), adds Ben's patch for another 8% speed improvement on JavaScript SunSpider regex performance, and properly launches the right number of threads for WebM decoding on multiprocessor Macs (four on the quad, two on dualies and one on everything else -- note that dual CPU G4s under 1.25GHz are still likely not to do well, just less badly). The pull I did was before Mozilla dropped a big load of patches on the ESR tree, all of which are small individually, but the sum total suggests we should have an RC and we will a few days before this is scheduled to come out (mid-March). I'm also going to start working on the 11 changesets over the weekend between doing my taxes (TurboTax still runs on PPC Tiger, which is awesome).

G3
G4/7400
G4/7450
G5

Server migration should be complete

2012-02-19T22:47:00.000-08:00

As mentioned, www.floodgap.com finally is moved off its old, dear Apple Network Server 500 (as someone on 68KMLA calls it, the "end table server") and onto the POWER6. This mostly went okay except for a glitch in the webserver which caused some downtime this arvo, but I think all that is fixed. Please report if anything is still not working: you should only notice that it's faster (we went from a 200MHz 604e to a two-way SMT 4.2GHz POWER6, which is quite a leap ;).

There is already some work on 10.0.3pre to report, including a fix for black images that don't show up, enabling additional WebM decoder threads on multi-core systems, and an encore from Ben to squeak another 8% or so out of JavaScript regular expression performance. More on that soon. Now I'm going to go put the cat to bed and veg out with the rest of this pint of pralines and cream.

Dangeresque 4: This time, it's Chemspill 2 (*)

2012-02-17T09:39:00.000-08:00

In a bid to release a chemspill just about every week, here we are at 10.0.2. This one wasn't Mozilla's fault; this was a serious flaw in a major library and everyone took this up the orifice, including Google Chrome. It also affects Firefox 3.6 and products based on it such as Camino 2.1, so you holdouts should update as well. Yes, we are vulnerable too. Because it would be more work to pull the new JS hotness out, and because I've gotten uniformly positive results from you, the beta testing audience, we're ~~gonna have to jump~~ shipping it in this release also. For that reason, the changesets are exactly the same.

This was supposed to include the "black image" fix (Mozilla bugs 720035, 689962 et al., tracked for us as issue 132) but the patch Mozilla currently has for review in M689962 dragged the browser down enormously and I backed it out before release. I'm going to try a smaller scoped custom patch that simply disables image optimization if the image is all black and this should not have the performance impact of the "bigger" fix. That will be in 10.0.3pre or final, depending. If this seriously affects you, you can disable image optimization entirely for now and this is explained in M720035.

In the "I told you so" department, since we're talking about security, the latest flaw discovered in Adobe Flash can be used in a platform-independent manner to steal account information from the browser using malicious scripting. The Power Mac version of Flash 10.1 is vulnerable, and can be used to exploit the browser without special considerations for Power Macs. There will only be more of these in the future.

Release notes and versions:

G3
G4/7400
G4/7450
G5

UPDATE: Just read that the Frieden bros. got a updated version of Timberwolf running on AmigaOS 4. This is a fork of Firefox (much as we are also a fork of Firefox) for PowerPC Amigas, mostly targetted at the (object of my lust) PA6T-based AmigaOne X1000, but works on SAM boards and the like. While they still have some work to do to get it up to current, it's good to see PowerPC-based Firefox alive and well on other platforms besides the usual suspects.

(*) Yes, I did watch a whole bunch of Homestar Runner and Strong Bad DVDs while working on this. Why do you ask?

10.0.2pre available, now with awesome, plus: more G5 optimization notes

2012-02-12T19:27:00.000-08:00

After the unexpected 10.0.1 release comes now the beta that was supposed to be 10.0.1pre and is now 10.0.2pre. This is the next big leap in JM+TI that reworks branches from the simplistic always-far calls in 10.0.0 and 10.0.1 to a set of intelligent algorithms that try to favour the branch prediction of either the G3/G4 or the G5, both written by Ben. It also includes Dave's square root routine, slightly altered for efficiency. The G3/G4 square root version improves square-root heavy code significantly over the old JavaScript stub function version; V8 Raytrace improves by almost 12% (the G5 has square root in hardware, but this is a very good implementation). If I may say, it's a nice example of why POWER ISA is, well, powerful. Besides using the fast reciprocal square root estimate instruction in the 603 and up, it also makes heavy use of the built-in FPU fused multiply-add to do Newton's method in fewer instructions. I might also add that x86 only just added FMA; AMD finally implements FMA (as FMA4) in Bulldozer, but Intel won't have what PowerPC had in 1992 until Haswell in 2013 (and even then only as FMA3). So there.

Ben's branchwork is the centrepiece of this release, however, and while it improves SunSpider by a modest amount, it improves loopy benchmarks like V8 by a huge degree. My quad G5 improves by about 45%, for example, and gets consistently around 900-950ms in SunSpider (down from around 1050). Our 1GHz 7450 G4 doesn't improve as much on SunSpider (2750 down to around 2600, so not quite at our AWOAFY? target), but still improves by about 40% on V8. Part of achieving this is splitting the way branches are handled into "big POWER" (G5) and "little POWER" (G3/G4) versions. Ben's original work did much what my code in our dear nearly-departed tracejit did, which was to have four-word branch stanzas padded with nops so that if a branch target was too big for a regular b[l] or bc[l] instruction (the normal relative branching instructions on PowerPC), we had enough room to turn it into lis ori mtctr b[c]ctr[l] which load the destination address into a register (usually r0), transfer it to the CTR, and then branch to the CTR (conditionally or always). This achieved 45% on G5 in V8 and about 35% on G4. SunSpider dropped down to less than 920 on the G5 as well.

So this was already a great start, but Ben's next brainwave was to free up the G3/G4's limited cache by reducing the branch stanzas further to two words, either branching to the target as usual, or if not possible, branching to a "trampoline" (not to be confused with the trampoline the JavaScript interpreter uses to enter JITted code, which is common to all and I handwrote in assembly language) in a construct called the constant pool, which has the actual far call. The constant pool is part of the JS runtime, provided by Mozilla normally for the ARM JIT where they dump constants to be referenced by the JIT code, but it doesn't have to be used for that. By doing it this way, Ben keeps more running code in cache, and as predicted, on the G4 this improved performance by another 3-4% in aggregate. (Ben later added another piece that only uses the trampoline when absolutely necessary, which fractionally improved this number further on G4.)

On the G5, however, this actually hurt performance and SunSpider climbed to a poorer result, nearly 1100ms; even with the later tweak to reduce trampoline usage, it was still around 970ms. Our theory is that the G5, being (in Apple's words) "very hungry, very fast and very sequential," pays too big an aggregate penalty to branch to an out-of-line branch stanza when a far call is encountered, for two reasons. First, it appears to be a smaller penalty (possibly even near zero given the aggressive ordering of the G5 dispatch unit) to have empty nops inline that take up some small proportion of instruction cache, because when those empty instructions are patched to a far call in-place the G5 does not need to introduce bubbles in its pipeline doing a branch into the trampoline just to branch again. In addition, the hypernerds amongst you will recall from our previous treatise on G5 optimization that there can only be one branch instruction in a dispatch group. The trampoline version must run in (at least) two dispatch groups, because there are two branch instructions, one to the far call in the trampoline and one in the trampoline itself, and both will each introduce a pipeline bubble of variable length. The far call in-place will still introduce a bubble, but the entire branch can in the best case execute in a single dispatch group because there is only one branch (the branch-to-CTR instruction at the end), and there will be only one bubble.

Because the G5 is really just a POWER4 with a deeper pipeline and AltiVec, this property is likely shared by later "big POWER" CPUs like the POWER5, POWER6 and POWER7, as well as "big POWER-like" CPUs such as the G5, Cell PPE and Xenon. We will likely have consumers that will want this branch optimization strategy, but we don't want to lose the gains we get on "little POWER" (such as G3, G4, e500, QorIQ, Gekko/Broadway and PowerPC 4xx) with the cache-saving trampoline approach, so we do both. On the G5, the original four-word stanza branching is compiled in; everything else (G3, 7400 and 7450) use the two-word branch stanza with the constant pool trampoline. The best of both worlds is thus achieved.

One final note on G5 optimization: I tested compiling the browser with 32-byte-aligned blocks and labels in the JIT allocator, and that slowed things down too (it is not obvious whether this can be more fine-grained). For that matter, when I tried building the browser with 32-byte-aligned loops, jumps, functions and branch targets, that too slowed the browser over the 16-byte-alignment it uses now. It appears to be all a balancing act.

10.0.2-final will come out at the same time as the ESR release. I also plan to write the debug only 11 fairly soon. Please note that I will be transferring service from the Apple Network Server to the POWER6 this coming (USA) holiday weekend, so there may be some intermittent weirdness the weekend of the 18th/19th/20th. In the meantime, please grab a beta build and give it a spin on your architecture:

G3
G4/7400
G4/7450
G5

10.0.1 chemspill

2012-02-09T18:12:00.000-08:00

Instead of the 10.0.1pre test release I was planning to make, Mozilla has identified a high-priority security and stability issue in 10 that we are affected by, so we now have a 10.0.1 available. All users are advised to upgrade. I also snuck in a fix for issue 129, since it was a low-risk repair. The faster JavaScript engine Ben was working on will be in 10.0.2pre once I have finished merging the G3/G4 and G5 versions together; they are different code paths in this version. We will talk more about that when it is available.

Waving from the fast lane

2012-02-04T13:57:00.000-08:00

I am typing this in the G5 build of 10.0.1pre, which has Ben's branch rework in it and Dave's square root (slightly modified). The G5 doesn't benefit from the square root, of course, and on SunSpider it only dropped a deceptive 100ms or so (from 1050ish to around 900 -- but this meets our target), but one test is, as usual, deceptive: Dromaeo rockets to a ridiculous 180+ runs/sec, and V8 is 45% faster.

I'm going to run 7450 and 7400 builds off today for conformance testing on the lab systems here at Floodgap Orbiting HQ. If they pass, then the last thing to do is switch the build snippets to run from the ESR and update the build documentation. If all goes well, you'll get to play with this new hotness this weekend or earlier. More strong work from our contributors!

10 converted to release

2012-01-30T21:01:00.001-08:00

As threatened, all update notifiers have been updated to reflect 10 to give our holdouts a chance to update while they still can. Also, I'd like to welcome Theo to our volunteer technical support squad, who has graciously volunteered to take part on Tenderapp. If you are willing to help enlighten the unenlightened as well (and file a real bug report now and then), send me an E-mail.

10.0 RC (finally) available

2012-01-29T22:16:00.000-08:00

The RC is finally available, after a few late-breaking bugs landed, and the G5 jammed most of the day to poot out these builds. The only difference (other than what Mozilla fixed) between this and the beta is that the help and feedback links now connect to Tenderapp, and some minor internals changes so we don't pick up Firefox hotfixes by accident. Speaking of, Mozilla did adjust garbage collection between beta and RC, and that seems a bit snappier.

I'll post more when Mozilla determines how they will maintain the ESR in their tree (no doubt as another repo). Meanwhile, 11 will be a changeset-only release because it is finally time to decommission the old Apple Network Server and I need the time to do so. Ben's changes to the methodjit branches and David's software G3/G4 square root routine will be in the next stable and unstable betas. Stay tuned.

Release notes and builds:

G3
G4/7400
G4/7450
G5

Waiting for RCgodot

2012-01-27T16:29:00.000-08:00

Ordinarily there would be RCs by now but Mozilla has not tagged the build branch yet and I know of at least one bug that may land prior to ship. Watch this space over the weekend.

Tenderapp support now available

2012-01-14T13:22:00.000-08:00

As I mentioned in the previous couple of posts, Mozilla will EOL Firefox 3.6 when 10 ESR exits the qualification stage (for the lifecycle of an ESR release, see the Mozilla Wiki proposal). As we are the unofficial overflow for PowerPC and 10.4 users that will be orphaned after 3.6 runs aground, we are likely to pick up a large number of new users in the very near future, and expecting Mozilla to continue to support them through SUMO is not fair on their resources.

Fortunately, Tenderapp offers free accounts to open source projects with an application, and I am pleased to announce that they have accepted ours. Starting today, users will be able to file feedback and questions through our own TenFourFox Support site. The Support link under the Help menu automatically points to this new site, there are Help and Support tabs on the main page and Start page, and the TenFourFox Help option will point to it with 10.0 final.

Please explore it and see what you think. Chris, contact me privately so I can give you a login to allow you to read and reply to support tickets. Other people who are interested in helping with the support effort, please request in the comments.

10.0b1 stable available

2012-01-08T14:10:00.002-08:00

Sorry about the RSS blip earlier; that was me hitting the wrong button. After a little more delay than I had previously planned, 10.0 beta is available, the first of our "stable" releases (based on the putative Firefox ESR).

Firefox 10 is actually a really strong release. In addition to the usual gratuitous UI change(s) (this time around it's the disappearing/reappearing forward button if you are using large icons in the toolbar), there are some remarkably solid additions in this release and my personal favourite is the Inspect Element feature. You can right-click on anything and select Inspect Element, and the HTML element appears in context with its children and parents, highlighted for your visual convenience. Even cooler is that you can dynamically alter the style, turn inherited and specified properties on and off, and find it within the page source. This is much like the old and much-beloved DOM Inspector (from Firefox 2.x and previous; still in Classilla and SeaMonkey) except that it is fully integrated within the main browser and not a separate XUL component. Great work, Mozilla!

Fx10 also includes 3D CSS transforms and properties. We render these in software (son of no-OpenGL-2-in-10.4), so performance is not great, but they work and they work surprisingly well, considering. Here is a static example from MDN: notice that all the 3D objects on the page are not images, but have selectable text, because they are in fact just transformed text. The proof of the pudding is in course in the animation, though, so why not a rotating HTML5 logo? Performs pretty well, even in software rendering on the 1GHz iMac G4.

Mozilla is also including a full-screen API where, if you permit it, a web page can dynamically open itself up to full screen. Now, if you're like me, you realize that the possibilities for phishing just jumped about twelve or thirteen times, and using it for full-screen video on pretty much any Power Mac is hopeless (my quad G5 even at full tilt can't push 1920x1080 WebM video in software). But it would be nice for presentations and other kinds of remotely-hosted demonstrations, and an excellent way to fool your boss that you're working (just switch to the tab with a static screenshot of Outlook taking up the whole screen :).

Finally, Mozilla has enough confidence in AMO to make extensions default to compatible. Now your addons don't get nuked every time you upgrade, unless, of course, the server says they are incompatible or they use binary or other unstable APIs. Those of you using the QTE will notice that it didn't get disabled like every other beta did. In fact, I didn't need to update any of my addons when I started using 10.0b1 personally. Nice!

Not all was wine and roses with the port. Besides breaking my keyboard over the weekend (and I really liked that keyboard, because it works well with all of my USB Macs even in OS 9), Mozilla upgraded the WebM libvpx to 0.9.7, wrecking our custom AltiVec work. This is not all bad because 0.9.7 is multithreaded and we support that on multiprocessor Macs, so this should help performance on marginal G4s with dual CPUs, and I was able to reconstruct most of our code in the end. Report irregularities.

Continuing in the bad news-good news department, tracejit is still in 10 but we can't ship it as default: SunSpider limps to a similar speed, but Dromaeo is 15-20% slower. I can't find a specific cause for this, and frankly if I try undoing things I'm likely to break JM+TI and JM+TI is the future. So the future is now. I am sufficiently confident in the performance and stability of JM+TI for it to now be the default. If you are using JM+TM, consider disabling tracejit entirely and turning on type inference before upgrading to 10 beta. I will leave tracejit in this release and the rest of the 10.0.x stable branch releases for those people who are absolutely convinced that methodjit is wrecking their machine somehow (it's not), but you need to be aware of the performance differential relative to 9, and it will not be in 11. We landed some microoptimizations and fixes in 10, but they are relatively minor and there is not much change in the numbers. However, Ben has some ideas on how to rework branches to more effectively make use of branch prediction and not make the backend insanely overcomplicated. We also have a new contributor who is working on a fast software square root for G3 and G4. This work will land on both stable and unstable, so there will be a beta for that particular stable release when it's ready.

Finally, something Mozilla did broke the way double and triple clicks select text. I think I have a workaround for it, but it might behave ever so slightly different from usual. Report non-trivial irregularities in issue 122.

Back in the good news column, Tobias' VMX text converters are patched up and back in with hardened algorithms, plus a new one for XPCOM text conversion, which should significantly restore page processing speed on G4 and G5. This completes most of the targets that support SIMD in Mozilla Core except for an obscure one that only deals with plugins, though we are always looking for others. None of these regress the infamous issue 84 and so I think we can call this a win. I also put in a minor fix to the backing JavaScript so that about:memory isn't a mess of unresolvable Unicode line drawing symbols.

It still isn't clear how Mozilla will handle the ESR rollout. My bet is that there will be a mozilla-esr repo or some such, from which we will pull for the stable releases. I am exploring low-cost ways of getting us an online helpdesk and support area for users so that we can eliminate our dependency on SUMO. Chris has volunteered to help with support. Other volunteers are requested. The beer offer stands.

For 11, we, and by "we" I mean "me," really need a break and this is the best one (closest to 10, so likely to have no major issues) to sit out on. I will still do a changeset for debug builds, but it's going to literally be just a placeholder, and there will be no binaries. If others want to take this one for the gap, go for it. Part of what I need the time for is to migrate www.floodgap.com off the 14-year-old Apple Network Server 500 that it still runs on to the big IBM POWER6 dual-core server sitting around idle except for being the gopher server and handling Veronica-2. There should be no service downtime like our unpleasant two months "network free" this summer because I'm simply going to transparently move the IP address to the new machine. I also really need to do finishing work on Classilla 9.3.0 and TTYtter 2.0 while I still have a chance. The next scheduled unstable branch release per se will be 12. I'm still planning to do it based on mozilla-beta if for no other reason than avoiding the small amount of perpetual churn on -aurora.

In the meantime, release notes and architecture builds:

G3
G4/7400
G4/7450
G5

Two forks diverged in a wood, and I, I took the one that had free beer

2012-01-04T19:06:00.001-08:00

(* which is odd since I don't drink, but it seemed better)

It's official: Firefox 10 will be the Extended Support Release, basically Mozilla's new legacy release, and will mean the end of Fx3.6 support on April 24, 2012 (assuming current release schedule).

The ESR will mean, finally, a stable base for long term maintenance. This is of great personal relief to yours truly, because keeping up with the Mozilla treadmill ("making the hippo dance") is not a lot of fun sometimes. It is also quite possible that Mozilla will use the ESR to cast off support for OSes they don't want to support anymore (even though more progressive elements are resisting this), and I would not be surprised to see 10.5 users forced onto the ESR along with other undesirables such as Windows XP SP1 and Windows 2000.

Right now the TenFourFox 10 port is mostly done and I have a debug build mounted which appears to basically work. I have not tried getting AltiVec builds up yet; I'm planning to lock myself in my office and do that this weekend when I'm not on call. Other than some annoying trivial internal changes, the biggest reason this port was delayed was that Mozilla changed most of their legacy NSPR booleans (viz., PR_TRUE, PR_FALSE and the PRBool type) to C++ booleans (viz., garden variety true and false). This invalidated around two-thirds of our patch sets and required a lot of manual adjustment. There was also a bug in plain methodjit that this unearthed and yielded several days of fruitless rooting around in the code until I found a working solution earlier today. The tracejit seems to basically work, but I won't be able to fully test it until I get opt builds running, so I have not made a decision about whether people will be forced to use JM+TI in this release (if tracejit has minor problems, I will try to fix them; if it's unacceptably crippled by Mozilla's type inference changes, however, it's probably not worth repairing). I have decided that we will use methodjit for the chrome, because this will cut back on memory and the chrome code is obviously not changing; the tracejit decision relates specifically to Web content.

Once 10 is kicked out to the unwashed masses, development becomes bifurcated. Unlike the regular ESR, which will only get high-level security and stability fixes much as the moribund 3.6 gets now, our stable branch (based on the 10 ESR) will get other fixes and possibly even new minor features. Most users will get this release and receive point updates to it, which should be reassuring to our change-averse general audience. If a major feature needs to be backported, there will be a regular beta-and-RC cycle.

Development of the main Firefox channel will still continue in the background on the unstable branch. Let's make this clear now for you pedants in the audience: the unstable branch is not intended to make release-quality builds, just betas and test builds. There will almost certainly be bugs and minor ones will not be fixed on any particular timetable; the point of the unstable branch is simply to keep the port viable in a general sense (and to give me and the other contributors a well-deserved break). Only beta builds will emerge on the unstable branch, not RCs. I may sit out 11 to give myself a break, and pick up with 12.

The next scheduled ESR will be Firefox 17. I think it's unlikely we get that far, but hey, I said we were dead in Firefox 5 and Tobias got the linker working; I said we were dead by Firefox 11 and Ben showed up with the macroassembler. So don't rely on my picks for the NFL postseason if you want to keep your money. If we make it to Fx16, then Fx16 will get a regular beta-and-RC cycle, and the stable branch will move to 16 and then settle again on 17. It would be a miracle if we made it to Firefox 24, though (it's doubtful Mozilla would still even support 10.6 by then, and we're not going to make a "Lionized" TenFourFox because that would make me nauseous), but we would still do security updates, of course.

There are a few other things I'd like you guys to think about. I have had some internal discussions with Mozilla and TenFourFox is currently their draft recommendation for Power Mac and 10.4 users still on 3.6 (always subject to change, of course, but this seems like their intent). Right now we piggyback on SUMO, Mozilla's support system, but this doesn't seem fair when Mozilla won't support PowerPC at all. Instead, we're going to need to start up our own user support and triage network like I already have set up for Classilla ("Report-A-Bug"), just on a much larger scale. I'm soliciting for someone or a group of someones to head up triaging user reports and helping support the influx of new users we will likely receive, translating user reports into bug reports as appropriate. Please E-mail me or nominate yourself in the comments. You will get free beer if we ever find a bar that would serve our PowerPC-using riffraff kind.

The second thing I'd like people to consider is what to backport in case we don't make it to Firefox 17 (or when we don't make it to Firefox 24), i.e., when we drop to feature parity. We would need to implement important CSS and layout features from later Mozilla releases to keep the browser core reasonably current, possibly new JavaScript features, and definitely some important extension-like functionality to incorporate into the interface and maintain ourselves (the QTE, adblock along the lines of Camino's, user agent switching to mimic later browsers, etc.). This is not a Christmas list and Christmas has already left the building; it is a "need to have or we lose significant functionality" list. I'm not soliciting for this list yet because we're still at source parity for the time being. If it looks like we fall off the wagon, we'll need to powwow.

I'm shooting for 10 beta next week. I'll post if there will be a significant delay.

AWOAFY? (Are We Old And Fast Yet?, or, 9.0.1pre JM+TI available)

2011-12-27T19:14:00.000-08:00

During the long death march to Firefox 4, Mozilla made much hay out of their Are We Fast Yet? site, a/k/a AWFY. AWFY was their daily dangling carrot while they worked on the first iteration of methodjit and offered regular feedback against the competition, and is still being used in the new type inference age. Well, JM+TI is here for PowerPC at last (that is, JaegerMonkey and type inference), and so is 9.0.1pre with new backends for your experimental pleasure. There are no other changes in this release and it is otherwise exactly the same as 9.0; it should fix all the known flaws in methodjit. (It does not include the fix Mozilla issued in Firefox 9.0.1 because I have received no relevant reports of crashes related to it, and it may impact performance, so there will not be a 9.0.1 unless I hear differently.)

So, with this release, I am now proud to bring you Are We Old And Fast Yet? where the first answer is yes and the second answer is "not enough." JM+TI now offers JavaScript speeds comparable to our old friend tracejit: on the quad G5, JM+TI (without tracejit) now matches the 1050ms on SunSpider that tracejit gets, and increases Dromaeo to 132 runs/sec. On our testing 1GHz G4/7450, Dromaeo rises to 38 runs/sec, and SunSpider drops slightly to 2750. (We're solving the missing square root problem for the present time by simply falling through to the JavaScript square root library routine. The G5 uses its native square root instruction, of course.)

This is already pretty darn good, but we can do better, and here is our goal for AWOAFY: quad G5 at Highest achieves less than 1000ms on SunSpider; 1GHz G4/7450 achieves less than 2500ms. These are totally doable; in fact, improvements in Firefox due to ship in the Fx11 timeframe may push these numbers down to target without us doing anything, but we're still going to shoot for low-hanging fruit optimizations in TenFourFox 10.

That brings us to what happens next. If Firefox 10 is indeed the ESR, as our Magic 8-ball predicts (it also predicted I would get a date with Scarlett Johansson, though), there are enough pieces of tracejit that it should still work and I think we should stick with tracejit as the default JIT for the duration of the 10 branch if so (with the option for JM+TI baked in for the adventurous): it's well tested, it's safe and it does well on our older hardware. I am waiting for beta 2 to be certified on mozilla-beta and once it is, we will begin the port. On deck for this release are some fixes from Ben and getting Tobias' fixed VMX text conversion routines back in action.

If Firefox 10 is not the ESR, then Fx10 will be the last tracejit release, and people will be forced to use JM+TI starting with Fx11. Sorry, this is Mozilla's call, not mine.

Either way, when we scramble to the ESR, the ESR will become our stable branch. Mozilla will backport security and stability fixes to the ESR which we will pick up, and unlike Mozilla, we will do other kinds of bugfixes to TenFourFox "ESR" that we deem appropriate. There will be no new features added, however. Most users will stay on this release.

For you, the rowdy, wild beta testers, there will still be beta releases, but not formal releases (because there won't be formal releases for awhile -- users will be kept on the ESR-based release). The situation we are trying to avoid is getting stuck on a release of Firefox that isn't getting security updates because we are unable to get the next one working. If, say, we get to Firefox 14 but can't make it to 15 -- and this is a real risk because Mozilla is already making noise about canning 10.5 support somewhere around Fx13, as we have previously discussed -- then we are marooning our users on a branch that will never get any fixes. By keeping later updates merely betas, most users are on a maintained branch and those of us out on the beta channel can just downgrade to the already existing release and live out our days in secure geriatric bliss in feature parity.

Assuming the ESR is 10, the next ESR would be Fx17. If we make it to Fx16, then we will do a regular beta release for Fx16, release Fx16, and go for 17. We can certainly backport security fixes from Fx17 to Fx16 if we fail at Fx17 and stay safe; the code is similar enough. If we don't make it to Fx16, then we can go on to feature parity against the ESR using the patches we've already done for later releases. So I feel pretty good about the future.

JM+TI is only transitional, however, because the future is Mozilla's new IonMonkey engine. This is months away from being functional, but I already looked at some of the code for the existing partially-functional backends and while much of our JM+TI work will transfer, there is a lot of work to do to get it working for PowerPC. Let's try to get as much wear out of the work we've already done.

Now, let's test AWOAFY. Download 9.0.1pre and go to about:config. Turn javascript.options.methodjit.chrome and .content to true, and turn javascript.options.tracejit.chrome and .content to false (yes, you are disabling tracejit; it is not compatible with type inference). Finally, set javascript.options.typeinference to true and restart the browser. Report your results! Remember to turn type inference off if you need to downgrade to 9.0, and turn methodjit off entirely if you downgrade to 9.0b1 or earlier.

G3
G4/7400
G4/7450
G5

Our own Christmas present

2011-12-25T08:24:00.000-08:00

After many days of non-stop debugging by Ben, Tobias and yours truly,

% ./jit_test.py --jitflags=mp ../../../obj-ff-dbg/dist/TenFourFoxDebug.app/Contents/MacOS/js # straight methodjit [1775| 0|1775] 100% ===============================================>| 257.0s PASSED ALL % ./jit_test.py --jitflags=mnp ../../../obj-ff-dbg/dist/TenFourFoxDebug.app/Contents/MacOS/js # JM+TI [1775| 0|1775] 100% ===============================================>| 270.0s PASSED ALL

Now, how did Santa fit all that down the chimney? Again, this is an unoptimized JS, so ignore the timings.

The bugs that were repaired to get type inference working are systemic bugs, so they may actually fix the failure cases for straight methodjit. For G3/G4, we are simply going to prevent the optimizer from using a square root instruction for the time being and just use the JS math library routine (and later develop our own optimized assembly version that can be inlined). I'm going to do some more internal conformance testing this week and if this all checks out, hopefully we will have a 9.0.1pre for you to test by New Year's.

And keep Chris out of the eggnog.

9.0.1 under review

2011-12-22T08:16:00.000-08:00

I'm reviewing the single change in 9.0.1 (backout of bug 708572) to see if it applies to us, although it probably does. However, the relevant crash (bug 711794) seems to primarily affect 64-bit x86_64 and we are of course 32-bit PowerPC, and there is a definite performance impact to taking the backout. We don't support the offending toolbars implicated in the STRs in any case. Crash reports matching the signature would be strongly appreciated to allow me to make the call (please don't send other crash signatures unless you are positive they are related somehow). There is no security impact to this bug that I am aware of.

Is Fx10 the ESR?

2011-12-20T19:00:00.000-08:00

I'm not sure if LegNeato made a Freudian slip, but comment 38 is very suspicious. It's also welcome news in case PPC JM+TI runs aground, because we could probably hack tracejit to work by itself in the ESR if we had to.

Two bits of bad news on methodjit. The first bit of bad news is that type inference requires the CPU to have a square root instruction, and only G5 does. We could emulate this on G3 and G4, but it would be expensive, of course. Assembly jockeys, here's your chance: come up with a routine in assembly to do PPC square root without a lookup table (you can use a couple constants if you like). It doesn't have to be fast because it's going to suck anyway; it just has to work. This routine might give you some ideas. Post your entries to issue 96; the winner gets beer.

The second bit of bad news is that we have some serious problem with how JavaScript stack frames are managed (not to be confused with the VMFrames we generate in methodjit), and it's not even in the code I wrote. I'm not sure if it's a compiler fart or not. This might be the explanation for some of the serious slowdowns on some sites, and it's possible the problem was there all along and it's just JM+TI that unmasks it. The problem crops up when nesting JS stack frames and if all else fails, I might be able to hack the interpreter to just turn TI off and drop to JM alone if the situation occurs. But that eliminates the entire advantage of JM+TI in the first place, and still isn't as fast as tracejit.

Life sucks. More later.

9.0 now release

2011-12-19T19:09:00.000-08:00

9.0 is now in the release channel and available to users. As stated in the previous blog entry, methodjit is not on by default for the unwashed masses, just the regex compilation with tracejit which so far has proven to be very fast and safe.

So far there have been no reported crashes with methodjit, but I have received at least two reports of significant slowdowns on selected sites. We are tracking these in issue 119. Please only report sites there that are clearly and reproducibly being degraded by methodjit (i.e., you turn off methodjit and the problem goes away; you turn methodjit back on and the site drags back down again), not general slow issues or slowness from some other cause. It would also help to isolate a script as the specific cause using NoScript or similar tools in case we have to minimize a test case.

Type inference is still being difficult, but the MIPS port does not require anything extra of significance for it to work and MIPS is "big RISC" like us, so it's clearly something I've done wrong I haven't found yet. Right now I'm investigating stub calls, since they are by design non-ABI-compliant, and the extant code may be just clever enough to get it working in some places but not others. It is also possible that these two problems may be related.

I have also made the executive decision that we will not advance to 10 until we have both type inference and the basic methodjit working with no showstopper failures (i.e., no reproducible crashes or long hangs that are not reproducible in stock Firefox, and composite benchmarks that are not significantly worse than tracejit). We will issue interim 9 releases until then, because odds are good tracejit will be in serious disrepair in 10, even though it still exists. When I have these two issues nailed out, there will be a 9.0.1pre for you loyal beta users to try.

9.0 RC available with secret sauce

2011-12-17T18:55:00.000-08:00

9.0 RCs are now available. The RC fixes the missing favicons problem (this is actually a Mozilla bug, but they will not ship the fix until Fx10, for those of you using Firefox on an Intel Mac) and a glitch with a malfunctioning chrome script that causes weird behaviour on AMO and certain other OpenWebapps-integrated sites.

Oh, yeah, the secret sauce. It is generally not my policy to introduce new code in an RC, much to the annoyance of our contributors, and although in a strict sense the new code is also shipping as part of regex compilation (which was in the beta) the "sauce" is turned off as shipped. However, we're also in a bit of a time crunch. Fx10 is likely to have integration problems with tracejit at best and outright break it at worst, and losing tracejit with nothing to replace it specifically means a real punch in our gut.

So we really need to test our new JavaScript PowerPC methodjit now so that we can start working on any bugs and get type inference working (I'll discuss what that means in a moment), preferably for 10.4Fx 10. And that's what you're going to get to do with the "RC," because the RC has methodjit secretly embedded in it! Start the browser and go to about:config, and turn both javascript.options.methodjit.chrome and javascript.options.methodjit.content to true. Do not change any other values related to JavaScript, and do not turn type inference on (the browser will crash immediately if you do). Leave the tracejit on. Restart the browser to clear everything out. The browser will be a little slower to restore itself because it will now start compiling the chrome methods with methodjit as well.

Tracejit and methodjit attack JavaScript just-in-time compilation in two different ways. Our old friend tracejit watches the interpreter run code. If it sees something it determines is "hot," it turns on a tracer that translates JavaScript operations into an intermediate language called LIR and from there into PowerPC assembly language. This is very fast and the fastest way of running certain kinds of code, but branchy code paths cannot be built with tracejit because the execution runs in too many directions, and an unexpected change from the normal execution path means the tracer has to start a new one. So that's where methodjit comes in. If JavaScript determines (through profiling and various internal heuristics) that tracing will be unprofitable on a section of code, it can try simply compiling the entire JavaScript method en masse. This is slower than tracejit at some tasks because it may end up compiling code paths it may never execute (most importantly related to covering the gamut of implied types JavaScript makes possible -- read on), but it also allows certain kinds of optimizations that a tracer will never see because it only sees "one view" of the script rather than the entire bulk of code. The browser will pick which of them is most effective, caching its choice and the JITted code, and you will notice that as more code is in the cache the faster the browser gets.

Tracejit has fundamental differences from methodjit and Mozilla is proceeding in directions that tracejit is not compatible with, which is why it is being disabled in 10 and outright deleted from the tree in 11. This is bad for us, because tracejit executes certain kinds of code far faster than methodjit ever will. Mozilla's solution to methodjit's overhead is to introduce type inference to JavaScript. Remember that I mentioned part of the trouble with methodjit is compiling unnecessary code to cover the possible (or at least reasonable) permutations for JavaScript parameters and variables. If it could be determined what the most likely set of types were for an execution path, then code covering just those types could be built and the rest could fall through to the interpreter for a second chance while the methodjit revises its guess. The guess has to be good, because the penalty is to lose both the code compiled and to have to execute in the interpreter, a penalty tracejit hardly ever has to take because the types are already determined for it by the time the tracer gets enabled (and it doesn't have to junk the code path it already compiled, because it might use it again). This is where Mozilla is putting significant resources to beat Chrome V8 and Crankshaft, and Fx9 is the first Firefox where type inference is being exposed to the user by default.

Type inference makes drastic changes to the JavaScript engine, and as a result, JM+TI (that is, JaegerMonkey/methodjit with type inference) is incompatible with TM (tracejit), even though you can combine JM+TM without TI. But because type inference and other optimizations are part of the future IonMonkey project, tracejit is now old news. This is sad for us because our tracejit is very heavily optimized, but that wasn't my call, and Mozilla was kind enough to give us until Fx11 to get our backend shifted over. However, we don't support type inference yet -- I couldn't get it fully working in time. This is just as well since we should really test the core browser without it first.

So, shipping in this release is the Nitro macroassembler for PowerPC (which is an .h file with certain semi-atomic operations and their assembly-language equivalents), expanded for Mozilla's purposes, and a lower-level assembler that the macroassembler calls to do the translation. These are the pieces Ben diligently worked on the first draft of, and this version includes my completed draft with the bugs worked out. They are not as optimized as they could be, but they do work, and we can delay that a bit to get any other major issues out. Both YARR and methodjit use the macroassembler, YARR for regular expression compilation (which we do have enabled in 9), and methodjit for JavaScript. There is also a snippet of code called a "trampoline" which is a skeleton function prologue, epilogue and stub call library I wrote by hand in PowerPC assembly to interface with the JavaScript C++ interpreter, and the other pieces needed for JavaScript to understand OS X ABI stack frames and register allocation. Those of you watching our stack usage problem will now note that our methodjit stack frames are a fixed 128 bytes, overall smaller than tracejit, and our YARR stack frames are a fixed 32 plus any subpattern usage. We should start seeing smaller stack overhead when tracejit goes away, which is a consolation prize. The interested can see what our stack frames now look like in js/src/methodjit/MethodJIT.h and js/src/methodjit/TrampolinePPCOSX.s.

There is an effort underway to port the TenFourFox PowerPC-specific code to Linux. The VMX code Tobias wrote will translate pretty much directly, but the JIT compilers will not because they assume the OS X application binary interface, not the Linux ABI. Linux ABI uses different stack frame sizes, a different linkage area (the first few words of the stack frame), different volatile register designations and different parameter register allocation. Tracejit was not really written for anything else but OS X, but methodjit is being designed to be friendlier to our PPC brethren on other operating systems like Linux and should be particularly easy to port to AIX (and a tip of the hat to Andrew, who is running server-side JavaScript on big-iron POWER servers like our very own POWER6 here at Floodgap Orbiting HQ). I don't have any ETA on this and I am not personally involved with the project, but I have had contact with them and given them advice and suggestions. I'm sure they will post when they have early builds to play with. Please note that TenFourFox itself will always be for OS X; this effort would be better understood as part of a PowerPC-enhanced generic Firefox.

So, how are the numbers? My quad G5 doesn't change much on SunSpider (around 1050-1070ms for both TM and JM+TM), but Dromaeo in 9 goes from 114 to 122 runs/sec with JM+TM. The 1GHz G4 in 9 goes from 32 to 36 runs/sec and from 3300 to 2800ms in SunSpider. Overall, while not the dramatic change that tracejit was, it's still progress and I expect even more when we get type inference running; the project goal is still to get the quad G5's SunSpider time under one second and we're almost there. In the meantime I want you to bang on the browser hard, please -- we've got to get all the kinks out!

Some housekeeping. Firefox rapid release is developing a long tail of people not upgrading to later versions for a variety of reasons ranging from ignorance to add-on anxiety to update fatigue, which they have attempted to fix with the 3.6 to 8.0 "major update." The issue of the uninformed and shortly-to-be-marooned PowerPC Firefox 3.6 user notwithstanding, we have the same problem, made more acute by the termination of plugin support with TenFourFox 6. Nevertheless, I think it is bad form to have so many users on an unsupported branch with known security holes and I'm considering updating all the version snippets to remind users that TenFourFox 9 is available for them too and to please update to a supported version. They could of course ignore this nag as well, but at least we tried. Argue passionately for or against this idea in the comments.

One last note: do not downgrade to 9.0b1 with methodjit on -- make sure you turn it off before trying to re-run the beta for comparison, or the browser will immediately assert. In fact, you should probably do that for any previous version of TenFourFox, just to be safe.

On deck for 10 will be Tobias' fixed AltiVec text converters back in the lineup so he won't come over to my house with an axe ;), and some other additional minor optimizations, but type inference is the big kahuna and I may delay the beta until it is ready. As always the Secretary will disavow, I mean, the beta port will start when Mozilla certifies the first 10 beta. And this blog entry will self-destruct in five seconds because of the secret sauce. So you'd better read the release notes and download for your architecture now. Good luck, Jim.

G3
G4/7400
G4/7450
G5

Hello, methodjit

2011-12-16T13:02:00.000-08:00

bruce:/home/spectre/src/bruce/mozilla-9b/js/src/jit-test/% ./jit_test.py --jitflags=mp,p ../../../obj-ff-dbg/dist/TenFourFoxDebug.app/Contents/MacOS/js [3544| 0|3544] 100% ===============================================>| 369.4s PASSED ALL

More about that when RCs come out. mozilla-release is now up to date, so 9 RC builds will commence later today, and if they pass conformance testing, up tonight or tomorrow. (Ignore the elapsed time as this was an unoptimized debug js.)

One other note. Some well-meaning individual submitted the QTE to Softpedia as a download. Please don't do that. It's an alpha, it's not finished, it has known bugs and we're not ready to support it yet. If it was you, please indicate to them that it was an error and retract it until we're ready with a beta. (In fact, you probably shouldn't submit it at all, since when it does hit beta we will just integrate it with TenFourFox itself, probably.) I appreciate your enthusiasm, but it was premature. Thank you :)

QTE 9 available

2011-12-13T16:27:00.000-08:00

The QuickTime Enabler for 9.0 is now available (alpha 114). As the version bump indicates, the only change from a113 to a114 is compatibility with 9. I am really hoping that Mozilla starts issuing simultaneous Add-On SDK compatibility updates with their betas, because it somewhat harshes our buzz to lose the QTE temporarily when we test our own betas. However, the QTE does not seem to cause any problems with stability, and because of this we may simply start including this as a built-in part of TenFourFox (I have not decided yet).

Simon Royal from LowEndMac has been trying to drum up interest in porting one of the open-source Flashes to PPC OS X. This is not a project I can personally donate time to, but if some of you out there are looking for a project the best one to port would be Lightspark (plus-minus Gnash for the older Flash applets), as it contains the most current version of the Adobe ActionScript virtual machine. The code is known to already work on PowerPC Linux, but the code would need to be altered to use either QuickDraw or CoreGraphics, and it would need to be buildable on OS X, so the port is possible but non-trivial. I'll add a carrot here: if someone gets this running and is able to maintain it, I will consider (technical limitations notwithstanding) lifting the plugin embargo. If the plugin exists, the plugin code in TenFourFox still works (we no longer test it, people are on their own) and someone is keeping it up-to-date, I'll not let it wither on the vine from this end. It's not much of a carrot, but hey, if you're just hanging around the house on the weekends now's your chance to be an abandoned platform rockstar.

Mozilla is due to sign off on Firefox 9 tomorrow, and we will do RC builds immediately after that. Tobias has fixes available for our VMX text converters which will appear in the beta immediately following. We also have a fix in tree for the icons problem, which is Mozilla bug 705516, so those of you using the image.mem.decodeondraw tweak mentioned in that bug should set it back to true before you upgrade to the RC or you will use more memory unnecessarily. RCs willl be announced here when they are ready, most likely by Friday or Saturday.

Methodjit is pretty much done, but we still have six tests (though out of over 1700) that fail and the browser is not able to stand up with the current code. I have not decided if I will hold Firefox 10 until methodjit is done -- tracejit is still in 10, just disabled. That buys us time for me to fix the last remaining failures by the end of tracejit in Fx11. Remember, don't try to enable it in Fx9 -- it will almost certainly not work, even though it is "secretly lurking."

9.0b1 available

2011-12-01T17:45:00.000-08:00

9b1 is available, corresponding roughly to Firefox 9 beta 4. This was a rather troublesome port -- Mozilla did some large internal changes in this release to facilitate type inference (which we do not yet support, stay tuned) and the architectural requirements needed caused significant delay in simply getting the browser to build. Even after that, in DEBUG mode it worked reasonably well but with VMX/AltiVec code paths enabled it crashed incessantly and I had to disable two of our text converters. I'm hoping we can get them reenabled for either 9 final or 10.

It also seems, based on comments in the previous blog entry, that there are still some sites with ridiculously large or complex numbers of scripts that still cause us to run out of stack. For 9, I've made a bandaid and backed the stack pointer all the way up to 0xf0000000 in memory, for a total of 1GB of stack. (On systems less than 1GB of RAM, it doesn't use this automatically, but it potentially could. In this case, it will get swapped to disk, so your system may thrash like crazy but it won't crash; the stack will then shrink back down after execution completes. For this reason, 1GB is now the recommended minimum, though we will still support 512MB machines.) Even with this "bandaid," this obviously doesn't mean we won't overrun it again in the future, and I'm unable to think of any way I can wring more stack out of the browser in 32-bit mode and 64-bit mode is not an option for any architecture except G5.

Methodjit will change this, but I have no idea if it will change it for the better or not. The key issue is that because of our very large register set and our requirement to abide by the OS X PowerPC ABI, we must save large stack frames to the stack, and our stack cannot dynamically grow unlimited. (This is a good thing -- such sites could destabilize the entire computer rather than just the browser if they were allowed to allocate stack forever.) Methodjit as currently written limits us to a much smaller register set, currently 16 GPRs and 8 FPRs of which all the FPRs and around half of the GPRs must be saved on the stack (plus the usual OS X ABI linkage area overhead). This is a smaller stack frame than tracejit uses, which can potentially use all nearly all the GPRs at once, but it may not save us because it may be easier for methodjit-generated functions to recurse with less checking (and that smaller set of registers in use may have performance impact, but I talked about this with Mozilla and it does not appear likely to change -- methodjit is limited to a total of 32 registers altogether). We are exploring ways to propagate an imminent stack exhaustion state to the browser so that a window like the "Unresponsive script" dialogue can be generated rather than, you know, crashing. If you have some ideas about that, see issue 114.

9 is the first installment of what will become methodjit, if we can get it done. This includes Ben's hard work on the macro assembler plus my additional hacking on YARR JIT to enable us to compile regular expressions, which was a gap we suffered from before. With regular expression compilation, the quad G5 running at full tilt drops its SunSpider numbers from around 1600ms to about 1050ms in 9. Yes, this is slightly slower than 8+YARR, probably due to some additional overhead in the interpreter, but it is still significantly faster than 8 alone. (As a side effect, methodjit compilation is enabled, but it does not work. If you were naughty and turned on any of the methodjit preferences in about:config, turn them off before upgrading to 9.0 or you will crash!) On sites like Twitter, it makes a big difference, and it will also accelerate certain extensions like Ghostery and AdBlock Plus. Think of it as a down payment on more JavaScript wizardry to come.

Even though we had to back out some of the AltiVec text conversion code, 9 also includes Tobias' AltiVec qcms colour management code and the completion of the AltiVec scale and colour conversion code, which improves WebM performance quite a bit. It also corrects a regression with JPEG decoding that Chris found and that Mozilla caused -- now JPEG AltiVec decoding is really really fast, and G3 is much faster than it was in 8.0.

9.0 makes a few small subtle interface changes to things like the go/reload/stop button, for really no good reason, but whatever. There are also additional HTML5 and CSS features, font stretching, and improved AJAX performance with a revamped, "chunked" XHR system.

So let's gaze into the crystal ball for a moment and consider some possibilities. Firefox 10 will be the last version of Firefox to have tracejit; it's gone in 11. Preferentially we will have our implementation of methodjit available for 10 beta and we must have it for 11, or we'll have to keep dragging a tracejit-able JavaScript to future versions, which seriously hurts our portability. We could really only fire a "gun" like that once, maybe twice, before changes in the JS API and other internal limitations make it impossible to merge an old JS into a later browser. The long and short is that we really need to have methodjit if we want to continue with source parity. If not, then we drop to feature parity and go from there.

This gets a little more interesting (and exhausting) in that Mozilla is now talking about dropping support for 10.5 in Firefox 13, which was inevitable. It's probably worth reviewing when Mozilla dropped 10.3 for Fx3 and then, of course, when they infamously dropped 10.4 for Fx4 which is why we exist in the first place. Like I say, it was inevitable for Leopard support to have its number come up so quickly; there are few reasons for Intel users to stay on it (Snow Leopard is faster and leaner and still runs PPC software), and Tiger was a much longer lived version of Mac OS X than Leopard was. It is also worth noting who drives each proposal to end support (hint: same person each time). Just sayin'.

The reason this interests us is that while there were many things in 10.5 that we could emulate in 10.4, there are many things in 10.6 (and, for that matter, 10.7) that we can't. Things like, for example, Grand Central Dispatch, or other hidden UI interfaces, or the graphics stack. Right now we simulate the dependency on CoreUI by using the old chrome code, CoreText with Harfbuzz and NSTrackingArea with cleverer management of events (and there are some 3rd party versions of it we could use if we had to), and 10.5 is limited to software rendering also, which makes things easier for us because we are also limited to software rendering and Mozilla has to support that. But it will be much harder when 10.5 compatibility goes away, because Mozilla will make assumptions about the capabilities of the class of remaining machines which many Power Macs will not meet, and will almost certainly try to leverage some of the new features available now that they don't need to worry about legacy support.

But wait, the plot thickens with a partridge and chickens. Most of us have already said our piece on rapid release (and there is some evidence that it is hurting Mozilla's market share already) and more reasonable elements within the Foundation have determined that having a long-term support branch might be worth it (the so-called Extended Support Release, or ESR). The ESR would be the second-tier release and users would be discouraged from running it, but it would be available for corporate or managed environments where rapid-release makes regular Firefox burdensome to manage. 3.6 is sort of filling that role now, so this would finally end updates to 3.6 and replace it with another long-lived branch, and then periodically update it according to a regular, slower timeframe. This idea is not new; the original ESR plan called to start around Fx8, and now it's talking 10, and in all likelihood I don't see much traction on it until the Fx12 or 13 timeframe.

Wait, did you say Fx13? The ESR is an important release for us: it guarantees the ground doesn't shift under our feet more than necessary and we get a certain degree of security patches. While Mozilla will discourage the ESR for regular users, there is nothing that says we can't build on it, and the ESR will be supported for at least 6 months, probably longer. So here's how the scenarios will play out. If we can hang on and make the jump until Fx13 (assuming that is the ESR basis), then we will be on a stable platform with security updates that we can continue to build on. If we develop past the ESR, and we are unable to maintain it, we could be marooned on a Firefox branch that has no updates (and has the requisite bugs from that release to boot). Nothing says that past the 10.5 cutoff date that we will still be able to keep the browser afloat.

So the current plan is to do whatever it takes to get us to the ESR and then start making official releases based on that, while continuing to generate beta releases in parallel off mozilla-beta as we have been doing. If it looks like we'll make it to the ESR after that, then we keep going. If not, we fall back on the ESR, no harm done, and backport features to that as feature parity. When the ESR ends, we enter security parity. That's still a lot of life left, and did you notice that TenFourFox has already had its one year anniversary? Break out the champagne and cheese, man!

As usual the beta will break the QuickTime Enabler because the Add-On SDK team doesn't have the foresight to get a working SDK to coincide with it in the Add-On Builder. Because of the tardiness of this release, the QTE will simply be a maintenance update, and I will post it when it is available.

Release notes and architectures:

G3
G4/7400
G4/7450
G5

Beta delay

2011-11-27T16:21:00.001-08:00

Well, 9 is finally up and running after some difficulty with the initial build. However, it's also having some real trouble being stable -- it crashes with only minimal use and I strongly suspect the new garbage collector. However, part of the plugin IPC stack also appears in affected backtraces (yes, with plugins off). I'm trying to debug this, but it's failing in subtle and unrepeatable ways.

If I am unable to solve the stability issue by next week, we will issue an 8.0.2 beta with the new features scheduled for 9 and decide what to do at that point. The ESR is not far away (Mozilla's extended service release, to replace 3.6), so that could give us an out. More soon.

Macroassembler ahoy!: more JIT in 9

2011-11-19T16:06:00.000-08:00

I'm typing this post in a modified version of TenFourFox 8 with regular expression compilation baked into the JavaScript JIT, and I'm already sad about having to go back to vanilla 8 for comparison testing because it's already making a huge difference for the sites I use. Regexes, for the uninitiated, are concise ways of expressing patterns to match against and/or extract portions from data. They obviously predate JavaScript by many years, of course (probably the language that is most associated with them today is Perl), but they are a common part of modern JS applications and as such their performance is tested by most benchmarks. Unfortunately, the tracer-based regex JIT that Mozilla wrote for Firefox 3.6 was removed in Firefox 4.0 for YARR (Yet Another Regex Runtime, from WebKit) and its own regex JIT, so we never got the advantage of JavaScript regular expression compilation and every prior version of TenFourFox fell back on the interpreter when a regex was encountered.

Well, no more. As part of our efforts to attack methodjit (more on this soon), we finished porting the Nitro macroassembler to PowerPC based on Ben's initial hard work and this enables us to use YARR. Better yet, we can use YARR JIT and still use our heavily optimized custom tracejit right now. So it's going to be in 9. This also eliminates our dependence on PCRE to maintain our regular expression performance because we can now use the same code as everyone else.

How much difference does that make? Well, it depends on how much that's in your workload, but on the quad G5 I develop on here at Floodgap Orbiting Headquarters, SunSpider drops from 1600ms to ... are you ready? ... 990ms. Yes, kids, we're already at our target of getting under a second on SunSpider, and we haven't even implemented methodjit yet!

However, it should be noted that this is because the part of SunSpider we consistently chugged on was the regexp portion, which was almost 650ms before, and is now about 45ms. Likewise, on V8, we improve from 627 to 769 purely on the basis of RegExp; on Dromaeo, which is a fairly balanced benchmark, we make a much more modest improvement from 110 runs/sec to about 119 runs/sec.

Thus on many sites you will see little difference, but virtually all sites with significant JavaScript requirements use some regexes and you will see some improvement on them, and some sites like Twitter use gargantuan expressions which now parse considerably faster (Twitter's fat regex size was, in fact, what caused TenFourFox 5 to break). It definitely improves browser chrome performance because large numbers of regular expressions are used by the browser's JavaScript code, and these regexes can now be cached ready to go in machine code. The downside is, like all JITs, they only pay off if they are cached and so there will be some additional memory demands (offset by no longer requiring us to cache PCRE results). So it's a net win, and G3 owners will be delighted to hear that this is not limited to AltiVec -- regular expression compilation will be enabled on all versions, including G3. And the good news for builders is that we were able to hack it to correctly compile on gcc 4.0.1, so no compiler change is currently required.

The news is not as good with methodjit -- we can't even get it to compile simple code, let alone run it, even though we know that our macroassembler works (because YARR JIT works). Mozilla is suspicious that we may have unearthed a bug with register allocation and I have some contacts who are hopefully able to give us some tips to debugging the problem. I'm still hopeful for the Fx10 timeframe, which is good, because it seems there are some regressions with Fx9's Type Inference (which does not affect the tracing JIT) and it would be better to have those shaken out before we try to implement that for PowerPC.

Otherwise, the Fx9 port is so far uneventful -- I'm about halfway through the patches and so far there have been no major issues, although we haven't tried to build it yet. There is a lot scheduled for this beta, including not only YARR JIT but also faster AltiVec text processing and AltiVec colour space management (encores from Tobias), so it's taking a little longer than intended. I'm shooting for Thanksgiving weekend, and we can all give thanks for that. :)

8.0.1 not planned

2011-11-15T13:17:00.000-08:00

Mozilla is chemspilling 8.0.1 this week, probably today or tomorrow, to cover bug 699134 and bug 700835. Bug 699134 affects only Windows, and bug 700835 only affects the most current release of Java on 10.6 and 10.7, so neither is relevant to us.

As for the 9 beta, I am waiting for a couple issues to shake out. When Mozilla marks beta 2, then we will pull and port. On deck are some more AltiVec ports from Tobias and some infrastructure changes. The methodjit port continues and can compile simple expressions, but YARR behaves badly or crashes. Ben and I are investigating this in more detail, and the possibility of forcing a compiler update to gcc 4.2 is being considered if it turns out 4.0.1 miscompiles YARR on PPC as well. Methodjit is still slated optimistically for the Fx10 timeframe; it won't be in 10.4Fx 9.

Just for fun, because I am a Forth nerd, here is a beautiful implementation of Jonesforth written in pure PowerPC assembly language. It still runs perfectly on 10.4.

Stay tuned for the 9 beta!

8.0 now release

2011-11-07T18:42:00.000-08:00

8.0 is now converted to release. Watch for 9 pretty soon; the port will start once Mozilla has certified the first beta.

I note with interest that Flash Player 11 requires 10.6 -- my 10.5 Intel mini could not update to it. (I'll put 10.7 on it eventually and grit my teeth for those few tasks I have that require an Intel Mac -- mostly Android development and Eclipse. 10.5 runs happily in VirtualBox.) So in not too long we will probably see Adobe abandoning 32-bit Intel Macs as well as PPCs on Flash; all the more reason to eliminate dependencies upon it.

8.0 RC and QuickTime Enabler alpha 113 now available

2011-11-05T14:49:00.000-07:00

Mozilla dragged its collective feetsies on Firefox 8 and didn't finally sign off on it until late this week, which is why this RC is so tardy. However, the G5 jammed all night (which was helpful because we had a winter storm, and it's keeping the office nice and cozy) and coughed out the release candidates for you the beta audience to bang on. As usual, reports of serious bugs appreciated, but most of the bugs I know about should be corrected.

Issue 84, the irritating table malformation bug, turned out to have been caused by one of our AltiVec optimizers (specifically for text fragments). This is a curious manifestation and one we would not have found in testing. We're going to be more cautious about further optimizations on this particular portion of code, but being the rash developers we are, more optimizations are nevertheless in the pipeline for 9. A fixed version of the optimizer is in the release and the parser bandaid can go into the rubbish bin where it belongs (whew!). Note that this means this bug never appeared on G3, because G3 uses the original C code.

Tobias is continuing his work on more AltiVec-specific optimizations and faster ones of what we've got. Many/most of his optimizations will appear in 9. However, priority one remains the methodjit and I'd like to make a public shout-out to Ben Stuhl who polished off the opcode work on it and it actually parses. It doesn't work yet, but this is super awesome stuff and he gets a beer too. I've got feelers out to another heavy POWER user who has expressed interest and if this gets off the ground, we may fork js into our own internal repo to speed collaborative development and merge this back into our usual changesets. The first draft of this will appear in the 9 beta changesets, although I doubt very much I will have it working by then. However, Ben's strong work puts us with a decent chance of having this ready by 10 beta, and Mozilla has given us until 11 before the tracer infrastructure is completely excised (a thank-you to Nick Nethercote and Dave Mandelin). More about that when 9 appears.

8 final still includes Tobias' patches for AltiVec JPEG decoding with ImageIO and faster AltiVec text processing. I also made a small tweak to the tracejit for a marginal performance benefit we weren't getting, and fixed the remaining theme glitches. This should cover all the major bugs. Those of you who hated tab dragging will be relieved to note that Mozilla has bowed to complaints and backed it out, but it will be returning. My estimated timeframe is somewhere around Fx10.

There is a simmering attitude of discontent with 8 (and to a lesser extent 7) that it has periods where it can drag or temporarily grind to a halt. I personally have not experienced this, but a fast G5 covers a multitude of sins and I know some of you have indeed seen issues like that which appear caused by some interaction with the database thread Firefox uses for history. This is not specific to us; it is a Mozilla bug and if you search Bugzilla there are several open and active issues related to it. It's hard to be maintaining a port of the browser and be the lightning rod for criticism of it when a good portion of perceived performance problems is systemic and not port-specific. I point out, for example, this comment (at the bottom); we'll use him as a public example since he has chosen to make his irritation public. Even though TenFourFox is a Firefox port, and does have its own unique issues, it is still overwhelmingly Firefox, and I think people should be willing to voice their concerns about the browser to Mozilla directly if at all possible. I can't be everyone's punching bag for a port where I control very little of the code, and I don't think there's any reasonable response I could make to a guy like that from this position (though I am reminded of this blog post). Unless someone comes up with Chrome for PPC, which has a huge hurdle and I know I've talked to people about that who have asked, you're stuck with Gecko, and Mozilla is already talking about dropping 10.5. Make it your own while ye may.

With that disagreeable topic dispensed with, let's go back to goodies. I've compiled the feedback on the QuickTime Enabler, most of it good -- people love it when it works, but it just doesn't cover enough of what's available. So for alpha 113 (yes, there have been almost 70 revisions between alpha 45 [7.0] and this one), we're concentrating on expanding site support and I've started with the big one, which is YouTube.

Let's talk a little bit about the technology behind the QTE so that the limitations can be understood. QTE works by grabbing video URLs (preferably to H.264 video) and stuffing them into QuickTime playlists which it then hands to QuickTime Player. TenFourFox is not involved in downloading the video or even authenticating to get the video; it's all streamed, played and managed by QuickTime 7. Because of this, videos that require session cookies (Vimeo) or authentication will never work with the QTE because TenFourFox can't tell QuickTime how to send that information -- there is no API to do so. For Vimeo, for example, you will still have to login and download the video, which fortunately you can do for free.

That still leaves many sites that don't require session cookies or the like, and YouTube is fortunately one of them. In this release of the QTE, you can now go to any YouTube video (that has H.264 video available -- most do), right-click on the page, and select SD, HD 720 or HD 1080. HD 1080, btw, will probably chug badly on anything slower than a G5, but a fast G4 should be able to keep up with HD 720. In fact, you can right-click on any YouTube embedded iframe and do the same. It works better if you have the HTML5 trial enabled so that you can see the video instead of a blank box, but it will still work. Try it on this (rather slick) Microsoft video reported by Engadget which has all three resolutions available. Just right-click on the video box at the end of the article, choose your resolution, and Cmd-F to enjoy a full-screen experience in QuickTime Player.

Some videos can't be accessed from YouTube in this manner and you will get an error if you try to. Sorry, I can't do anything about that. I have encountered exactly two in the 50-odd videos I have tested, though. This will also not work for embedded objects -- i.e., those sites that embed the actual Flash applet rather than the iframe, which is the preferred method. I am thinking of a way around this, but it is likely to be convoluted.

This version also corrects an issue with corrupted characters in the destination URL dialogue box and improves performance slightly. Also, the issue with the video not automatically starting on some systems is in fact a QuickTime settings issue. If the video does not play automatically (give it some time to buffer, please!), go into Preferences in QuickTime Player and make sure that Automatically play movies when opened is checked.

To install alpha 113, make sure that alpha 45 is removed (it should be, it's not compatible with 8), and download the .xpi. Drop it on TenFourFox and it will install. No restart is required; you can play videos immediately.

I would like other suggestions about services to add in the comments. These services should be popular, widespread and have known ways to access their video streams. As I mention, some will not be compatible with this method -- Vimeo is the most notorious.

Well, go to it and have fun. TenFourFox 9 beta is probably about two weeks away. Release notes and architectures:

G3
G4/7400
G4/7450
G5

Showstopper alert: the end of tracejit

2011-10-19T10:01:00.000-07:00

We're tracking Mozilla's determination to disable the tracing JIT for JavaScript (bug 693815), which would count as a serious blow to us and would stop us updating to a later version if we weren't able to get the method JIT operational. JavaScript compilation is our biggest marquee feature and we absolutely have to have it functional. This bug has already landed for 10, but the switch-off itself is relatively trivial to back out. What won't be trivial is what will follow, because it will almost certainly break the tracer even if it is reenabled.

Methodjit, which is the JavaScript method compiler, is about 60% written, but this is just a first draft; it is in no way tested to even build, let alone function. It may force switching to a newer compiler at a minimum, though Tobias has already done this work and we are already on a different linker. However, what I'm more concerned about is that I won't have it either stable enough or fast enough by the time it will be our only choice as a compiler, causing regressions at best and wholesale loss of functionality at worst. Those of you who are longtime loyal users already know it took us until almost 7.0 to get most of the bugs worked out of our current tracer, and that was handed to us almost finished and already in use by Adobe (by the way, Adobe has accepted our finalized tracer into nanojit). This is being developed from scratch, is a much larger undertaking and is being forced to occur in significantly less time.

Job one and the best case is that we get the method compiler backend (this includes the assembler and the modifications to the method compiler for the PowerPC ABI) in time for 10 beta; I just don't see this happening in time for 9 and 10 is still iffy.

If we don't get this ready for 10 beta, the current outlook (which can still change, because 10 is the nightly) is that all that will land in 10 is disabling the tracer and investigating any regressions. This is pretty straightforward to undo, but it serves as a warning. Once it is accepted that the tracer will be disabled by Mozilla, then backlogged incompatible changes will start landing, and I hope they land in 11; if they don't, then we will either back them out or force a downgrade to a known working version, most likely the release from 9. We can only maintain this hybrid for so long, obviously, before it becomes incompatible with the JS API the browser requires. If we can't reasonably complete the methodjit and we fall behind, then we will drop to feature parity and source parity will end. Optimally we would like to conclude source parity on a release that coincides with a Long Term Support branch because we will then have the advantage of Mozilla security updates, but we can provide these ourselves if we don't get that fortunate.

If you know someone experienced with PowerPC assembly who would like to contribute, please advise. I'm pretty good with PPC machine code, but I'm no genius, Surely someone from Armonk reads this blog and has some copious free time?

Meanwhile, we are continuing on for 8. For final I have approved Tobias' improvement to the UTF8 to Unicode AltiVec converter that is noticeably faster, and because this is a frequently utilized function that everything calls, everything speeds up. For 9 we will add accelerating colour management and additional encoding methods with AltiVec. Even if JavaScript stalls, the rest of the browser core is advancing at a rapid pace. I expect Mozilla will declare the RC within one to two weeks, and you will get it shortly afterwards.

The QuickTime Enabler unfortunately will not be so timely. Mozilla's Add-On SDK group has stalled on 1.2, which is the SDK required for 8, so I have not been able to do more work on it. I do have some surprises in store and I hope to have this available very shortly when the Mozilla Add-On Builder supports the 1.2 SDK. And I thought that Jetpack add-ons were supposed to be insulated for this ... We'd have done better with a standard XUL extension! We may do this anyway when time comes to integrate QTE directly in to the browser, but we're not up to that point yet.

More soon.

8.0b1 available

2011-10-07T14:36:00.001-07:00

TenFourFox 8 beta 1 is available, corresponding approximately to Firefox 8 beta 2. Fx8 is probably the first "major" update to Firefox since 4; the rest have been mostly merely incremental. Not that there haven't been important features introduced in 5, 6 and 7, of course, but 8 has some really nice new features. The one I really dig is the moveable tabs a la Chrome. Instead of grabbing tabs and having a hard-to-see placeholder mark your drop target, the tab moves with you. If you pull it out of the window, it turns into a thumbnail so that you know it will generate a new window on landing. It's really nice, and since it's really nice and awesome, it naturally may get turned off before release because it's apparently breaking some add-ons relying on the old tab behaviour. That would be a serious bummer and I hope it doesn't happen, because it's very elegantly implemented. It is also accompanied by smoother tab animation and the ability to load tabs on-demand when you start the browser instead of all at once. It's great work.

You're going to see the other major new feature immediately when you start the browser, and that is the new add-ons check panel. Each new version of Fx (and TenFourFox) will inventory your add-ons for you each time you update, allowing you to disable ones you don't need or forgot you had and see which ones won't make it. This is also a very welcome feature.

And, at the same time you experience the new check panel, you will see the major change we have introduced ourselves with this release: the end of appearance parity. Mozilla significantly rewrote large portions of the widget library with this release, requiring a large amount of code to be converted or reverted. Buttons looked wrong, the browser chrome was super dark and didn't match even Leopard's darker window style (and was totally bogus in Tiger), and some controls didn't even appear at all. But, unfortunately, this is expected; Mozilla dropped 10.4 compatibility and I am very surprised they are still maintaining 10.5 compatibility -- I expect that to end probably around the end of Fx3.6 -- so we should not suffer any illusions about the rug not getting pulled out from under us. With 8.0, by policy TenFourFox's interface will be at feature parity with Firefox, not appearance parity. You should still be able to do the same things with the interface, but colours and rendering may be different. We will eventually spin off the widget and theme ("Pinstripe") packages into separate Tiger widget and "Tigerstripe" themes to preserve compatibility and insulate us from sudden shocks and changes. This will probably be completed when Mozilla ends 10.5 support.

As a result, there were several appearance bugs that did not appear until late in testing and I have chosen to release the beta with them so that the rest of the changes can be banged on. Please don't report these, I already know that 1) there is no button frame on the buttons on the first-run add-on panel (I will probably just convert this to a regular Cocoa button) and 2) the Places/Bookmarks forward-back buttons have the rounded corners on the wrong edges. They will be fixed in the next beta and/or RC. I will also redo our easter egg because even though it is a long-time crowd pleaser going back to the Classilla 9.0 days, it is also probably rather tasteless in light of recent events. Charles Moore put it best: "I didn't always agree with Jobs, but where I didn't, I am obliged to concede that he was more often proved right, or at least reading market prospects and trends more accurately than I." RIP, Steve.

The lighter window chrome will startle some people. This is not a bug; it is on purpose. I never liked the darker chrome on Tiger and the code Mozilla added in 8 causes mismatches with the tab bar in 10.4, so I wrote new colour standards for Tiger to match standard window chrome and tested it on Leopard as well to make sure the colours are continuous. (Issue 16 still applies if you have the tab bar disabled.) This will be the new chrome for the time being, though I am considering emphasizing the tab edges to make them stand out more visually. When we fully split off Tigerstripe, I would love to have contributions (please, not merely "I want" suggestions: write the bloody code) for customizing our theme uniquely. If the Mozilla mountain won't come to Mohammed, then Mohammed is going off to Aruba to hang out with gorgeous co-eds. Your CSS can power those co-eds. And that was probably the worst metaphor I've ever written in this blog.

Oh, one other neat thing in Firefox 8 is HTML5 context menus. Right click on the animated image in this demonstration. I think this has loads of possibilities for application designers.

So that's the big overview, but wait, there's more! Tobias has written an interface to allow the browser to use ImageIO's AltiVec-accelerated JPEG decoder, which improves overall rendering of JPEG images on G4 and G5 by almost 125%. We scrubbed this for security issues in 10.4 and 10.5 and we can't find any; even if something does pop up, it's easy enough to switch back to the slower but safe in-tree JPEG library. We are only planning to do this for JPEG; PNG has known security vulnerabilities and isn't AltiVec-accelerated anyway. G3 users will use the regular JPEG library as before. Another beer for Tobias! Tobias has also submitted a very simple change to reduce the browser's CPU usage when it is in the background; it was continuing to receive mouse events unnecessarily and trying to process them, and issue 19 covered the circumstances where it actually needed them, so the code to receive the unneeded events was removed. Laptop users will particularly appreciate the battery impact. Please report any variances in activity.

We are also using minor tweaks to the AltiVec code introduced in 6 which increase performance marginally, and there is another change I need to land in the tracejit before RC/final to correct a small performance problem there too.

For those concerned about issue 84, the infamous table shift issue that plagues some pages in 6.0 and 7.0, there is a bandaid in 8.0 which fixes the reported sites and test-cases. The bug is due to an optimization problem and does not appear in unoptimized debug builds (by the way, for builders, debug builds are now fixed in 8.0 and should be working again). I still have not isolated the exact place where it occurs, so there is now a very grotesque parser patch that rewrites the page code to eliminate this specific issue. I tried to minimize side effects as much as possible and the parser bandaid will log occurrences to the Console to facilitate study. The patch does not appear yet in the changesets because I am hoping I won't have to ship with it, but if I do, it will be there and I will be sad.

Chris is continuing to work on our localization infrastructure and translators are wanted. Most of the Firefox strings apply, but slight changes are required for our few interface differences. If you are interested in the localization effort and/or working on the locale installer, as always, sign up in issue 42 and issue 61. Chris drinks his beer in several different languages.

The future of JavaScript is methodjit and I am continuing the conversion on an internal tree. This is the method compiler for JavaScript that Mozilla is shipping as JaegerMonkey and will use for the forthcoming IonMonkey. I am really pushing this as a priority because it also accelerates regular expression parsing; even though we successfully got bug 684559 accepted in the tree rather than contending with the sucky YARR regexp interpreter, we suck wind on SunSpider's regex testing and we can gain some big time back by precompiling them. My goal is to get the G5's SunSpider time under one second (right now it is approximately 1600ms on my quad 2.5GHz G5 in high power mode). I really want to have this ready for testing by 9 or 10. Watch for more announcements.

Some housekeeping. I have not heard any objections to running ads on this blog to allow me to recoup some of the hosting costs, so I will probably set that up. (You are free to AdBlock, of course.) Also, here is a little ranty bit, not for our normal blog denizens but for newcomers here with an axe to grind. I continue to receive letters ranging from petulant to nasty about TenFourFox and plugins from people who don't read the announcement on the main page, on the start page, or in the release notes. You don't pay me for this browser, you don't contribute to it, and you don't have to use it. Ignoring all else about plugins and PowerPC, if your life online entirely consists of Flash movies and Zynga games, get a new computer. There will never be a Flash 10.2 on PPC. There will never be another version of Flash on PPC, ever. This policy reflects that reality. Sending me E-mails on how you hate what I've "done to Firefox" makes you look entitled and selfish, and is a waste of your time and mine. The Apple store is over there. Do yourself a favour and buy a nice new mini, and don't use TenFourFox. Above all else, don't complain to me about it. Ranty bit over.

Anyway, that wasn't directed at you guys, the loyal supporters. You get the good stuff. Read the release notes, then grab for your architecture:

G3
G4/7400
G4/7450
G5

7.0.1 under review

2011-10-03T08:05:00.000-07:00

Mozilla has released a chemspill for 7, but not for the bug you would expect (viz., the BEAST SSL issue we talked about last blog entry). Instead, this release covers bug 680802 and an unusual case where simultaneously trying to update the browser and an add-on causes the add-on to be apparently lost (in reality it is simply hidden and doing operations in the Add-on Manager will "resurrect" it).

After reviewing the bug, this appears to be another case where our more limited functionality saves us again. Although I can think of some scenarios where you could get a simultaneous update, because TenFourFox forces you to download a whole new copy of the browser for updates due to our more limited infrastructure you will rarely be updating both add-ons and the browser at the exact same time; instead, you update TenFourFox, it downloads new versions, and life goes on. It is possible to trip the bug, but you really have to work at it and I'll just let you think of how that might occur. I have only had one report of Adblock getting waxed updating to 7.0, and the user was able to manually restore it afterwards.

Because point releases are expensive for us and because I believe the risk is low, the current plan is not to release a 7.0.1. If you are one of those few unfortunate people who did manage to figure out how to louse up the browser and your add-ons, Mozilla has an add-on (ha!) that will temporarily patch the issue. You can download the add-on repair tool from Mozilla directly. Please report any irregularities.

8.0 beta is currently running in debugging mode internally. Mozilla, damn them, has significantly rewritten large portions of the widget toolkit requiring some extensive backouts and patching to work on 10.4, so we will likely drop parity shortly with widgets. The browser, however, basically works and I hope to have the beta out by the end of the week. There are some great TenFourFox-specific features in it too. More about that soon.

BEAST smothered by security blanket

2011-09-27T13:17:00.000-07:00

7.0 was converted to final this morning and I'm pulling down 8 beta 1 for initial conversion as we speak. This will require more work than 7 because Mozilla committed a number of 10.5/10.6-specific widget changes which will need to be rewritten for 10.4, but I'm pretty confident this is possible. You'll be the first to know.

A number of people have privately asked me whether TenFourFox is susceptible to the BEAST SSL attack revealed over the weekend. This rather startling hack was able to decrypt a Paypal cookie sent over SSL, allowing the attacker to assume the identity of the logged-in user. It should come as no surprise that 1) the Mozilla security team has been discussing this (along with representatives from Google, which uses the Netscape-originated Network Security Services library also) for several months, and 2) as shipped, TenFourFox is not vulnerable. Notice that I said "as shipped." The exploit demonstrated by Rizzo and Duong used a Java applet to do the server thrashing needed to get the key out, and TenFourFox disables all plugins including the Java Embedding Plugin, so the applet cannot run. It may be possible to do something similar in Flash (though such code has not yet? surfaced), but obviously this doesn't ship enabled in TenFourFox either, and the version of WebSockets in Firefox 7 and TenFourFox 7 cannot be used to trigger this exploit.

Even with plugins, Java and/or other suitable footholds available, the SSL flaw in question is not trivial to exploit: not impossibly hard nor prohibitively expensive, but neither is it simple, quick or easy. An attacker must have control of the network the victim is using (such as a compromised or open wireless link), and must be fast enough to spy on the connection and manipulate the victim's browser to try combinations on the server the victim is accessing before the session terminates. Under the optimal conditions of the demonstration the "victim" was pwned in about two minutes; real world assaults will undoubtedly require significantly more time. It is also possible for servers to immunize themselves against the attack by negotiating ciphers that are not vulnerable to the flaw Rizzo and Duong are taking advantage of; RC4/ARC4 is one such cipher and Google is using it already on its services.

The best solution is to leave TLS 1.0 behind, but for a variety of reasons this has yet to happen, and perhaps this will be the impetus to change to a safer, later TLS version. Once a final fix is decided upon for NSS, there may or may not be a 7 chemspill; we'll follow Mozilla's lead here. A similar fix will be included in Classilla, which is technically vulnerable but hard to exploit because OS 9's MRJ is mercifully too old and Flash 7 does not support sockets of the type required.

Super omnibus update: 7.0 RC, updated QuickTime Enabler, and Floodgap is back up

2011-09-23T21:35:00.000-07:00

Well, heck, let's start with the good news first. AT&T installed the T1 on Monday and DSL Extreme activated it earlier today, so Floodgap and its associated projects, including TenFourFox.com, is back online. Point your friends, enemies, rather less good enemies of theirs and friends of those people's less good enemies to it.

The T1 comes at (naturally) a substantially greater hosting cost; though it is amortized against using it for my other projects and as my regular means of Net access, it's still a pricey (but very smooth and low-latency) way of getting hosts online compared to DSL or cable, neither of which were practical options for reasons many of you already know. As a policy matter I do not accept project donations as this has the risk of encouraging expectations I and the other contributors can't satisfy, but I am considering allowing Blogger to put ads on this blog to make a little of the money back. I won't object if you Adblock them away, of course.

Mozilla has signed off on the 7 release candidate, so now we have our release candidate available ahead of Firefox 7's formal launch on September 27th. As with previous release candidates, if there are no showstoppers then this will become the formal release. There is one small fix in this version (in addition to Mozilla's fixes): because Floodgap was down there was nothing to post update snippets against, so this fix adds the 7-specific update checks. As a result, there is also a new set of changesets for builders. Read the release notes and grab for your architecture:

G3
G4/7400
G4/7450
G5

Also available, as promised in the previous blog entry, is a new version of the TenFourFox QuickTime Enabler. If you are new to the QTE, please see this blog entry for basic instructions; in short, the QTE allows you to play certain media types outside of the browser in QuickTime, including H.264 video (which Firefox can't play at all) and most other audio and video files by right-clicking and selecting Open Media in QuickTime. In alpha-45, here's what's new:

Updated for Gecko 7 (alpha-43 will not work with 7.0)
Overly long URLs don't cause garbage characters in the URL dialogue anymore; they will simply be truncated (but will still play)
If you are not offered the QuickTime option initially, start to play the video. When the video appears and starts, right click again and see if the option is offered then. If it is, and you open the file in QuickTime, the enabler will now automatically stop the video in the browser (if possible) so that you aren't playing it in two places. This should enable JavaScript-based video code that tries to detect browser capabilities to also be playable in the Enabler.

Download QTE alpha-45 and drop it on your browser window to enable it. It does not require a restart. Remember, this add-on is still in testing and development.

"One more thing." Many of you will have read the press coverage about the "BEAST" SSL chosen plaintext attack demonstrated at ekoparty this weekend. Virtually every browser is vulnerable to this; Opera has already released a patch and Chrome has one in testing. Note that Mozilla and Chrome share SSL code (i.e., they both use NSS), so the same fix is likely to appear in Firefox, but it is not currently in 7. There is substantial argument about how easy it is to pull off in practice, but certain things can make it easier, and we might talk about this once Mozilla goes public with a final determination.

Man, it's good to be online again.

7.0b1 available, whither 6.0.2, and a special pirate edition: YARR, THIS INTERPRETER SUX

2011-09-04T12:31:00.000-07:00

So, it's Labor Day weekend and I've spent it watching Law and Order DVDs, painting the house trim and tweaking TenFourFox. Today, for those of you with nothing better to do on a holiday weekend either, 7.0 beta 1 is available, roughly corresponding to Firefox 7 beta 4. Generally I am loathe to talk about subjective speed improvements in this blog because they are just that, subjective. However, there is a lot to like about Fx7, and the biggest one isn't the memory footprint improvements (because we're forced to take a hit on that, read on). Instead, it's the graphics stack, and the new Cairo really benefits our rendering pipelines. Pages paint quicker, animations are snappier and movie playback is less jerky. The improvement is subtle, and there is still much improvement to come, but I think virtually everyone will notice the difference. Tobias, the early trailblazer on 7, said it was the first version he felt was truly useable on his PowerBook. While I politely demur that I've been perfectly happy with TenFourFox on my iBook G4 ;) , I certainly do agree that 7 is better in nearly every metric.

Also new in 7 is the beginnings of a new HTML5 canvas backend (Azure, which will eventually take over the entire rendering pipeline; fortunately for us, the CoreGraphics version is likely to work and even if it doesn't there's still Cairo), improvements to Sync, a new Telemetry module for reporting performance back to Mozilla (we don't track this information ourselves, but if you want to remind Mozilla that PowerPC lives, go for it :), a new API for sites to determine load times and adjust themselves to slower systems, and CSS3 ellipsis support.

Oh, and it takes http:// out of the menu bar. Strangely, I'm not as offended by this as I thought I would be, but I'm not wild about it. gopher:// does still appear, though, and so does https://, so I guess that's all that matters. :D

So, yeah, about that "memory footprint" thing. Firefox 7 is being touted as the first product of Mozilla's MemShrink project, bragging numbers that reduce its overall memory usage by 20-50%, will leak considerably less, and will more aggressively release memory as tabs are closed. Naturally such must be taken with a grain of salt, but I will say that memory usage is noticibly less on both my G5 and my iBook. In typical usage on the iBook, comparing 6 versus 7, 7 seems to use about 25% less memory, and the system is able to reclaim it more easily.

Our problem, however, is it appears we are dramatically underestimating the PowerPC's JavaScript stack demands. Longtime TenFourFox veterans will recall several bugs that in succeeding order jack up the stack ceiling more and more, and in this version we jump from 64MB of stack to 256MB of stack (repairing issue 85). This is unfortunately unavoidable because the PowerPC ABI requires us to save registers to the stack on function calls and the PPC has a massive number of registers. As OS X has a hard cap limit we must ask for this cap in advance, and the original limits we asked for were too small because our stack frames are so much larger, causing it to crash out on certain scripts. This means in the worst case the browser may require a minimum of 256MB of RAM just for the stack, let alone the app or the rest of the OS. For this reason, starting with 7.0, the minimum system requirements will be for 512MB of physical memory. Systems with 256MB of RAM will no longer be supported. Every Power Mac we support in TenFourFox is capable of at least that much; even my Outrigger beige Power Mac 7300 has a gig, and my blueberry iBook G3 has 576MB. The recommended minimum will now move to 768MB. Fortunately, thanks to the other improvements, you'll be able to do more with the rest of your memory, and the 256MB is only a peak possibility, not a persistent requirement. So do yourself and your Mac a favour (RAM is dirt cheap these days) and max out your memory now.

A few of you will be wondering about that cryptic title, and that's the other thing that we need to mention: YARR sucks. Mozilla is expanding their JIT to JavaScript regular expressions using WebKit's YARR library. YARR originally used the Perl-Compatible Regular Expression library (or at least their heavily hacked version of it) as the interpreter fallback for when a JIT was not available, and since we don't (yet) have support for methodjit or the Nitro assembler on PowerPC, we used PCRE for regexp evaluation. In 7.0 Mozilla "upgraded" YARR to a version that now uses its own interpreter as the fallback instead of PCRE. All of their shipping platforms use the JIT, which to be sure, is very quick.

We knew about this going in and several of Tobias' patches were to fix the JavaScript core to deal with the situation where the YARR JIT was not enabled. When I got this together and ran it through Dromaeo for conformance testing, however, I was shocked to see the G5 drop from 110+ runs/sec (in 6) to a dismal 65. All of this regression was paid on the regular expression part of the benchmark. When I hacked PCRE back into the JSRegExp portion of the library, the number went back to the old figure. While WebKit admits benignly that YARR's interpreter is "slower" than PCRE, frankly we really take it in the shorts with YARR and so will any other tier-3 platform that doesn't yet have methodjit/macro assembler support. I filed this with Mozilla as bug 684559 so they can ignore it, but we are shipping with PCRE back because we can't afford a hit like that. Eventually we will have methodjit and we can play in the full JIT party too, but that won't be for a couple cycles yet.

Specific to us, because of my continuing downtime (AT&T is coming out to look at installing a second NID this week for my T1), there has not been any new TenFourFox-specific features in 7 other than updating Sluggo for the new Apple CEO. However, as mentioned, issue 85 is repaired, and a 10.5 specific bug with dropdowns (issue 72) has a putative fix. This fix is limited to 10.5; if you are one of those affected by that bug, please let me know if triggering dropdowns actually crashes the browser instead, because this would imply that my theory is incorrect (i.e., that issue 9 should only be worked around on 10.4). 10.4 is not affected by this issue. This version does not fix issue 84, the table spacing issue; although we have test cases, we haven't narrowed it down to the exact Mozilla patch that provoked the issue and I don't want to wreck my one working 6 repo until I don't need it anymore since none of my ad-hoc offline repos have history. As this bug is a simple nuisance, occurs only on certain sites and the sites in question are otherwise useable, we're shipping with this bug for now.

Soon we will need to turn our efforts to methodjit so that we can keep pace with Mozilla's improvements to JavaScript. If you know PowerPC assembly language and want to help, we can always use more coders.

I'm also working on a new release of the TenFourFox QuickTime Enabler. Watch for it in the coming week or two.

In news from the Mountain View mothership, Mozilla is planning a chemspill for their chemspill, viz., a Firefox 6.0.2. This version "fixes the fix" for DigiNotar, which seems to have caused some collateral damage, but their fix-fix is similarly controversial. I am not planning to generate a 6.0.2 as 6.0.1 will fail safe and I'm not convinced there won't be more changes in the pipeline before 7 emerges. If you are affected by the changes in 6.0.1, you could always use this beta, of course. :) However, if there is a large demand for it, I will consider releasing it as well.

For now, read the 7.0 beta release notes, and grab for your architecture:

G3
G4/7400
G4/7450
G5

6.0.1 now available

2011-08-31T08:41:00.000-07:00

6.0.1 is now available. As mentioned in the previous blog entry, this repairs the issue with fraudulent SSL certificates released by DigiNotar. Please note that all previous versions of TenFourFox are vulnerable to these certificates, so you are strongly urged to upgrade. There are no other changes to this version, and as such, there is no new changeset pack for builders (the old 6.0 one continues to apply to the current release repository). Read the release notes, then grab for your architecture:

G3
G4/7400
G4/7450
G5

6.0.1 chemspill imminent

2011-08-30T17:53:00.000-07:00

Mozilla has decided to release a 6.0.1 chemspill due to a nasty issue with more fraudulent SSL certificates being issued through a compromised certificate authority (DigiNotar). Unlike the earlier issue that brought us 4.0s back in the day, this one actually requires the entire root certificate to be withdrawn because it is believed that the CA is completely pwned (rather than just a matter of blacklisting a few rogue certs). These malicious certificates are believed to already be in the wild, and there's apparently quite a few as Google Chrome's bad certificate count is up by 247! Mozilla is tracking the issue as bug 682927, but it is currently sec-locked pending release.

Obviously this is a serious enough compromise that we will also be issuing a 6.0.1 which is being built on the G5 as you read this. (Classilla users, this fix will be ported to 9.2.3 when I can get the G4 MDD system back online; unfortunately Classilla development is stalled entirely until my connectivity is restored.) I plan to have 6.0.1 available either tomorrow or Thursday. Although issue 85 has a fix, I do not plan to have it ride along with this release because it may increase memory pressure on marginal systems. More about that when the 7 beta emerges, which I plan to work on as soon as Mozilla certifies beta 3 (probably this week also).

Decisions on 7.0

2011-08-23T14:36:00.000-07:00

6.0r is mostly a success, with the majority of users pretty happy with the improvements. We do finally have a reduced test case for issue 84 (the table rendering problem which was reported by some users), but I don't have a cause for it yet since my debugging was limited to what I could do pretending to buy stuff at Starbucks and hiding in the back leeching off the WiFi. With this smaller and fully independent test case I hope to be able to do a more work to that end on the debugging G5. I'm also thinking about grabbing 5.0.1 and making a internal-only build to see if the bug was there too.

On the Floodgap connectivity front, AT&T should hopefully have the T1's NID in by the end of this week, if DSL Extreme's rollout schedule is in any way accurate. Then it's just a matter of running the line into the server room and getting the CSU/DSU up, and we should be good.

Once the T1 is in and we're back in business, I intend to start on Fx7 immediately, which as I mentioned earlier is mostly done already due to Tobias' hard advance work. If it's not, then we might try the emergency repo trick again that we used for 6.0 final, but I'm really nervous about a beta that's going to have no testing at all except for what Tobias himself has done, and I won't have full Mercurial history for debugging. Either way, I intend to deal with this no later than the end of the month. I am hoping to land a few riskier fixes such as a putative fix for issue 72, and put issue 78 to rest at the same time; I also have an idea to deal with user reports of intermittent and inconsistent JavaScript-related crashes, but naturally such bugs intrinsically lack stable test cases and the fix is a bit drastic, so I haven't decided if I'm going to try it or not. (I do have the fix written up, but without a test case, I have no idea if it actually accomplishes what I think it does.)

6.0r available

2011-08-16T08:01:00.001-07:00

Thanks to you sharp-eyed people, issues 81 and 82 have been repaired in TenFourFox 6 (that means printing and print preview should now work correctly again in 10.4, and progress bars shouldn't shimmy around in 10.5), and the fixes are now spun out as 6.0r. I really appreciate everyone's patience with the server downtime and the delay in getting this out, but we've still met the release date, and we're still at source parity with Firefox 6!

To reiterate for people new to this blog, since this is now the "update channel" until I get Floodgap and my business line up again (I just signed a T1 contract with DSL Extreme today, so let's hope that works out), this is the intended final version of TenFourFox 6.0. If you are still using the 6.0 release candidate that came out on Saturday, please download and replace it with 6.0r. The version will still be 6.0, but about: should show a build ID of 20110815. Since there are no other showstoppers, this will replace 5.0, and there will be no subsequent 5.0 or 4.0 releases.

As one more reminder, plugins are disabled by default, and are no longer supported. If you are new to this policy, refer to previous entries in this blog, or see this wiki entry on plugin support.

For builders, there is a new changeset pack up to apply to mozilla-release with the issue 81 and 82 fixes in it.

Please read the revised release notes, and then download the version for your architecture:

G3
G4/7400
G4/7450
G5

TenFourFox 6: saved by the sneakernet

2011-08-13T15:43:00.000-07:00

UPDATE: retracted due to issue 82, respin available tonight or tomorrow

I had a brainwave (well, more like a duh moment) over the weekend about how to sneakernet the Firefox 6 repo updates to the G5 build system, which is still marooned from the Internet. (A parenthetical note about that: I've sent Time Warner the beginning pieces of a lawsuit and basically told them I'm walking away from the contract unless by some miracle the line's in this week. I'm quite certain it won't be. I'm getting bids for a T1, which I probably should have done in the first place; with luck, the downtime won't be more than a week or two more, and then Floodgap will be back in business with a super duper faster line.) So the iBook made several trips to the local Starbucks and back to the house, and several hateful cups of Zen tea later, TenFourFox 6.0 RC/final is built and this post is being typed in it.

I'm a little nervous about this release; there's been pretty much no time for serious conformance testing on all the architecture builds, although the iBook G4 checks out and so does the G5 (what I can test locally). So please thrash it. There's no new features in it, just fixes. The changesets for beta 1 still apply, so there is no new changeset package. If there is a serious showstopper, I did do a cursory security review of Firefox 5, so we can build a 5.0.2 for security parity purposes if needed.

For users new to TenFourFox 6, remember: plugins are disabled (see previous posts).

Looking ahead, Tobias (bless him) has already done just about all the porting work on Fx7 in advance and he says it seems to run really well on his G4. I have his changesets on one of the marooned servers, so as soon as I can snatch the Fx7 repo once it goes beta, we will work on that. I am quite concerned about Fx8, though. Mozilla has landed several changes to the build system that expect builds to occur on 10.6 (even though the build is ostensibly linked against the 10.5 SDK) and obliterate PPC references. This is unwindable (though it will probably hit the vanilla PPC builders such as MonkeyBoy and El Furbe), but may have other side effects, and a simultaneous change to require and use CoreUI for drawing UI elements is also of serious alarm to me as it means our Tiger widget code may no longer work even if we restore the old code. They seem to be having various issues with it, however, as you might expect from an Apple private interface, and I'm hoping they give up and punt it to Fx9.

Watch for a re-release tonight or tomorrow Pacific. ~~Read the release notes, then grab:~~

G3
G4/~~7400~~
G4/~~7450~~
G5

Time Warner: still sucks

2011-08-08T13:56:00.000-07:00

We're still down and Time Warner Cable Business Class just sent out their third site survey to say, "we'll call you." I've had it with this company. Don't ever do business with them. I'm preparing a complaint to their legal counsel this week; this is unacceptably bad.

The problem is that this means our build system has no access to the Net either, so it can't pull down code and that makes building another 6.0 beta problematic. Unfortunately for us, Mozilla is still planning to release Firefox 6 on schedule, issues with Fennec permitting. One option is to build an interim security release "5.0.2" with whatever patches I can cobble together and then sneakernet the binaries off the G5 so that at least we maintain security parity. If our downtime extends up to that point, then we might try that. However, you'd have to watch this blog since obviously the automatic update system doesn't work either. I'm really hoping they get it together soon, because although I could request for and grit my teeth over a T1, it's going to be itself at least two weeks installation and I can't even start on that as long as I have an enforceable contract with Time Worthless.

I have never dealt with a company this awful, and I'll be doing my darndest not to renew with them when the contract is up. They are truly abysmal. If this is how they treat their business customers, I'm sure many of you have horror stories about how they treat their residential ones.

More as I have more information.

tenfourfox.com/floodgap.com temporarily offline starting 31 July

2011-07-30T12:55:00.000-07:00

Starting tomorrow, Floodgap and all domains hosted on it (including tenfourfox.com) will go dark thanks to my new business ISP, Time Warner, who promised me two weeks for the buildout which has since ballooned into four weeks, beyond my contract with my current provider. Allegedly the buildout will be complete by the end of next week; during that time www.floodgap.com and gopher.floodgap.com will be dark, but you can still access the Google Code project and this blog, of course. E-mail sent during this time will fail over to my mail peer, but I will not be able to receive it until I'm back online.

I apologize for the downtime and I'll be happy to post the rep's cell number if this takes longer than a week, since I have a signed contract with them and I'm pretty steamed.

2 6 4 H what browser do we 'preciate

2011-07-20T15:48:00.000-07:00

So Lion came out today and as expected it only serves to make Macs stupid. So maybe we were the smart ones for sticking with PowerPC after all. Still, Walt Mossberg likes it, in the way he likes most things Apple puts out, as I found out from watching his H.264 video at the Wall Street Journal. ... Oh, snap! Did I not tell you? Let's watch H.264 videos from TenFourFox! Not even Firefox can do this trick!

Because Flash or Safari are obviously the only ways to watch H.264 video online from a Power Mac, and we're trying to jettison Flash, an idea I'd had kicking around for awhile was to enable a QuickTime window so that you could watch the video, at least, even if it wasn't composited into the browser. While fiddling around with 6.0, I realized that Firefox's js-ctypes module could allow the browser to do this kind of manipulation in JavaScript. Better still, it could be implemented as an add-on so that people who want to play with it can. Even better still, Mozilla had a supported example of feeding images to iPhoto which I could modify to feed videos to QuickTime. It just needed to be updated to be a restartless Jetpack, which I've done too.

The trick is QuickTime Media Links (.qtl), a little-used feature of QuickTime classic -- but notably not QuickTime X -- which lets you build lightweight Internet "media shortcuts" and feed them to QuickTime Player, which then goes and fetches the video online for you. So this add-on is, in essence, quite simple: select a media element (video, audio and images are supported), determine its source and write up a temporary .qtl file, and then feed that to QuickTime Player. QuickTime handles all the streaming and playback. No mucking around with AppleScript, just "clean" CoreFoundation code. In practice it's not quite that simple, but we'll talk about the limitations in a bit after you've had a chance to play.

Download the add-on. Don't install it yet. Remember, this is alpha software. It has not been well-tested on 10.5 or on a large number of sites.
These make good demos: either Walt Mossberg's Lion video review, which won't play at all in TenFourFox, or this HTML5 video test, which will normally play WebM in TenFourFox (but has H.264 as an alternate). Open it up for the standard behaviour to see what it does without the add-on.
Drop the add-on on the TenFourFox window. This extension is only supported for TenFourFox 6. I do not plan to support it in 5 or 4. You will be prompted to confirm its installation. It will install without a restart.
Now, right-click or Ctrl-click the video. There's a new choice: Open Media in QuickTime. Click it. A dialogue will tell you the new destination (mostly for debugging purposes). Then QuickTime Player will open, and ...
OMG!

The extension works for images, HTML5 audio and HTML5 video (although audio is not well tested yet). It will scan the available choices and prefer MPEG-4 (or MPEG-4 in an MOV container) since MPEG-4 can be hardware accelerated in QuickTime; otherwise it will go with what it can get. Images work too, mostly for purposes of demonstration. I'm considering making it prefer MP3 audio, but haven't decided about that yet.

Here's what doesn't work:

WebM video for some reason won't play, even with Perian 1.2.2 installed, and I'm not sure why (QuickTime objects to the video format). I'm not really too concerned about this since the browser can play it, but it would be nice because it would expand the number of machines that can handle WebM video. I'm wondering if its media sniffing code is failing and I have to hint it a bit more.
Autoplay doesn't work, even though I'm asking QuickTime to do it.
Not all elements are detected by Firefox's context menu code as qualifying selectors, so some video elements will not offer you the option. I may need to simply ask for 'everything' and then filter it myself rather than relying on the browser.

I don't plan to make this a part of the browser for awhile, mostly because it needs a lot more testing and because even when it will be working it's still somewhat of a hack. Still, it does work, and gives us another way to keep our Power Macs supported with modern Web video. Please give it a try and let me know your thoughts. Remember, this is very alpha and very unfinished.

I have not heard much noise from Mozilla about the next 6.0 release. I am presently in the process of moving to a new house, so I will not able to generate builds for about a two-week period starting next week as the G5 is in transit. With luck this should coincide with the 6.0RC, since there have been no major problems reported thus far. Your patience is appreciated.

6.0b1 available

2011-07-14T20:41:00.000-07:00

6.0b1 is now available, based on Firefox 6.0 "beta 2" internally (Mozilla, realizing the liability that rapid release is becoming, is trying to play down these numbers but ultimately I think that horse has already left the barn).

People like 5's speed; 6 is not IMHO appreciably faster, but does not appear appreciably slower either. It is a bit more busy, however. Recent patches to garbage and cycle collection have calmed this down, supporting my theory that more aggressive garbage collection is the reason, and on my G5 5.0 used around 3% CPU at baseline whereas 6.0b1 now uses around 6-7%. This is less on my iBook, which went from 2% to about 4%, but still higher. Fortunately this does not seem to translate to poor responsiveness, and the browser in my informal measurements does seem to be hanging on to less memory, so at least it's achieving its purpose. The big changes for memory usage, however, will not emerge until Firefox 7.

As mentioned, 6 adds mostly under-the-hood features again, chief amongst them Mozilla's personal flavour of WebSockets; more HTML5 features; media query support; and some handy but unspectacular developer tools. There is also about:permissions, which doesn't really do much for me compared to the (much faster) Preferences tab and isn't exposed in the UI which implies it's not doing much for Mozilla either. Still, these are necessary changes, and at least they're out in our hands faster, so rapid release is at least achieving that goal even if it's cheesing off the rest of the known galaxy over the version number creep in the process.

There are some slight improvements to JavaScript performance due to code reorganization, but most of this is barely above the statistical noise threshold and probably isn't noticeable in practice.

In the bug fix department, 6.0b1 fixes the issue with 10.5 and Java that Apple deliberately caused (see bug 668639), as well as Twitter's site redesign that causes the browser to stall out on fat regexes (issue 77) and cut down on spamming the console log with debug information (issue 70).

Also, I managed to get ahead of schedule with my move and sit down with some custom code. Specific to 6.0 are two AltiVec-enhanced changes for text and HTML processing: a VMX text fragment scanner, which improves dynamic operations on pages by 10-15%, and AltiVec-accelerating UTF-8 to Unicode conversion. Because OS X usually hands us 16-byte aligned memory allocations, even though we have no equivalent for the SSE2 unaligned store code in that routine we're still usually able to use the AltiVec fast path.

In addition, a double-pumped 128-bit algorithm is now available for AltiVec WebM YCbCr conversion, improving over 5.0's algorithm by 30-40% and translating to a real 5-10% improvement in video playback. Speaking of, Mozilla is futzing around with the buffering algorithm again. In 5.0, Mozilla would hold video in favour of audio, even if that meant stuttering the video. In 6.0, Mozilla will "pump video," even if the audio can't keep up, more along the way that 4.0 used to do it. I prefer the former, but I know some people prefer the latter. There is no easy way to switch between them and I bet 7.0 will change this around some more, but have your say in the comments.

Speaking of video, this is what you will now see by default if you go to a Flash-enabled player or application:

That brings us to plugins. Plugins are disabled, as warned. By default, this is what happens now:

Sites will think you have no plugins installed. This is the correct behaviour, because that allows the site to present you with alternate (hopefully HTML5) content.
If the site is hard-coded to use an applet, object or embedded plugin instance, the message above (or one like it) will appear. If the element is positively identified as a plugin, the "More information ..." link will take them to this wiki page on the subject.
Both about:addons and about:plugins will show no plugins installed; about:addons will explain why.

Without rehashing earlier arguments on the subject, plugins aren't being maintained for PowerPC, aren't secure, already can't access some content requiring later versions (such as Flash 10.2 and 10.3), already have bugs, and are being changed in Mozilla in ways that will likely make them incompatible anyway. I assume that if you read this blog, you are already aware of all this. It is not my intention to specifically break plugins, but I want it clear that I won't support them further and people use them at their own risk. Furthermore, the browser must ship in a safe configuration out of the box rather than expecting people to install things like Flashblock (which I heartily recommend). That said, plugin code is an integral part of the codebase, so there will always be plugin code in TenFourFox. There's just no guarantee that it will work, and 6.0 already has at least one change that will break compatibility with certain older plugins.

Regardless, I did promise you that you can still opt back in. Before you do, understand the following caveats:

The browser will always ship with them disabled. Your previous setting will stick, of course, should you choose to opt-in, but by default they will not function.
No bug in plugin support, even a serious or crippling one, will ever hold a release back. While I may try to fix bugs if I have some spare cycles (as if!), I make no guarantees or promises.
Application issues with plugins in operation, even crashes, may be WontFixed as bugs if other issues have precedence or for any other reason. However, bug reports with patches are welcome -- strongly encouraged, even. If you want something fixed and you can fix it, contribute. I've got enough work to do just fixing all the stuff Mozilla breaks in 10.4.
Finally, plugin code may stop working at any time without warning. I'll reasonably endeavour to prevent this, but if plugin code is preventing the browser from functioning I reserve the right to stub it out entirely to get the rest of the browser to operate, and even if it does compile it may be utterly bogus.

The relevant plugin settings are in, natch, about:config. There are five specific settings related to plugins:

The master switch is tenfourfox.plugins.enabled, which by default ships set to false. In this mode, the browser pretends as if there are no plugins at all and all of the other settings are irrelevant. You can dynamically switch it to true and reload the page to instantiate the plugin, but it's best to restart the browser afterwards after flipping the switch to true to make sure that all of your plugins are re-enumerated properly.
Embedded plugins are controlled by tenfourfox.layout.hideplugins. This ships also set to false, so embedded plugins are not hidden by default, assuming plugins are enabled. If set to true, this only prevents the plugin from starting up and operating; a placeholder will still appear on the page (usually blank space), and the page will still think that the plugin is operational in most cases. This setting is the riskiest one to enable, as it can facilitate drive-by instantiation of plugins.
Scrolling plugins is controlled by tenfourfox.layout.smooth_scroll_plugins. This ships set to true, so that any instantiated plugin in that window will force smooth scrolling on to reduce graphical artifacts. This can be annoying on slower systems, so for them you may wish to set this to false.
Full-page plugins, such as QuickTime objects and PDF files, are controlled by tenfourfox.layout.hide_fullpage_plugins. This ships set to false, so full-page plugins are not disabled by default, although due to painting bugs you may need to interact with them blindly to get them to start. If set to true, the plugin will simply not start when instantiated in full-page context, although it may still work embedded (such as QuickTime).
The Java plugin is controlled specifically by tenfourfox.layout.hide_java_plugin, which ships set to true and applets requiring the plugin will not start (though other applets may). However, depending on your system, the Mozilla Java Embedding Plugin may not be what you want anyway; you may want the Apple provided Java Plugin. Remember that Java is deprecated and will become unsupported by TenFourFox when 10.5 is unsupported by Apple (which will be very soon now). Check that your applet works without plugin support before messing with this option, as certain Java applets will crash the browser within the plugin context.

For the record, I am putting my money where my mouth is. I'm watching my videos in MacTubes or over WebM, I view my PDFs outside of the browser, and listen to music files in QT Player and watch Flash videos with Perian. Security must come first, even at the cost of functionality or convenience. I hope you will agree, and even if you don't, I hope that you'll appreciate I've tried to meet you halfway with this back-door option.

Release notes and builds:

G3
G4/7400
G4/7450
G5

5.0.1 = 404

2011-07-12T21:57:00.000-07:00

While I finish rebuilding 6.0b1, a parenthetical note that there will be no TenFourFox 5.0.1. We aren't affected by the font issue on 10.7, obviously (and the code is riskier on 10.4), and the 10.5 Java issue can be worked around using the Terminal tips in bug 668639. Since releases are expensive for us, a single workaround-able issue is not enough to force a new version on our end. While I will revise the 5 changesets to deal with the font issue changes, that will occur after the beta comes out, or they can be applied to the original 5.0 source tarball instead of the current hg tree tip.

Sweet 6 point oh een (or something)

2011-07-07T18:11:00.000-07:00

The reason it's been quiet here lately is, of course, that I was working on porting Firefox 6.0, which hit the beta channel on Tuesday, and there it is. In fact, this post was written up in it.

6.0 is the first full iteration of Mozilla's rapid release strategy (i.e., a full 6 weeks at each stage), and yet, if Firefox 5 should really have been Firefox 4.1, this still feels like 4.2. Out of the box, the only major user facing feature is "highlighting" domain names in the URL bar (really, dimming the rest of the URL). This is another Chromism and I'm not really very fond of it personally. Mozilla is also introducing, as a trial balloon, a Permissions Manager accessible under about:permissions. However, if you're like me and have a lot of specific cookie permissions, it takes forever to load and is not significantly better than the old way with preferences; it's also not exposed in the UI anywhere.

Under the hood there are additional HTML5 elements now supported, most notably <progress>; WebSockets (which were pulled for security reasons in Fx4); enhanced developer tools, and MatchMedia support. And, well, that's about it.

The build is not going entirely smoothly. Tobias Netzel has shipped me a 64-bit version of the ported Xcode linker we use while I try to get a debug build mounted, but optimized builds do function, obviously. However, the debug build is exhausting the memory on the 8GB G5 buildhost, so this appears to be a 32-bit limitation. We may have to add a build requirement that the debug build be built with the 64-bit version of the linker, which would limit the machines that can build it, unfortunately. More on that as we sort this problem out.

Specific to us, I'm phoning this release in because Mozilla landed a lot of crap that tripped up porting our changesets. Among the new changes was splitting off some of the text handling features in the Cocoa widget code (bug 519972), which naturally upset the Tiger customizations I had made. I believe I've successfully dragged back all the 10.4-specific code that we need into the new places, but I'm still not convinced everything is correct. This took several days to rewrite, and we will have another set of rewrites to do in Firefox 8 related to it. On top of that, Mac OS X 10.7, which went golden master and is expected to emerge "any day now," has serious bugs in its implementation of ATSUI (bug 663688, currently sec-locked), which we must use in 10.4 because 10.4's secret Core Text support is incomplete. Mozilla thus rewrote much of the legacy font handling code we rely on in terms of Core Text, which blew up the compile on 10.4. I've hacked this back into submission, but it will break again because that whole section will be revised when 10.5 support is dropped from Firefox and I am unfortunately all but certain that will occur somewhere around Firefox 9, early next year. After all that work, and trying to move to a new house, I really haven't had time to work on more TenFourFox optimizations for this release. We should get a break in Firefox 7 (if we're still able to build it, of course), so if we get Fx7 to stand up as 10.4Fx 7, then I'm going to work on some more AltiVec improvements at that time.

That brings us to plugins. Mozilla has made the first change which will break some legacy plugins in bug 468678, which was a long-warned threat to disable resource files as a data exchange method in Mac plugins. Cursorily Flash and QuickTime don't seem affected in their recent versions, but people using old versions might not be so lucky, and other plugins depending on this will break. This is the first change of many that will erode plugin operation, so as threatened, this version turns plugins by default off and support for plugins ends. I have wired up a preference to reenable them which will be posted here, as I assume if you follow this blog that you are technically versed enough to understand they are no longer supported, may not work correctly, may not work at all in the future, and already have known bugs. More about that when I get 6 beta 1 polished for release, hopefully next week.

On the 5 front, Mozilla is ruminating about a 5.0.1 release to address the 10.7 issue I explained above. This is not decided yet, but my insiders tell me it's probably going to happen and would be a Mac-only release fixing that bug only. This does not affect us right now, but because it has the font changes in it, it requires us to backport our fixes for 6 to 5 which could be iffy if we need to build more versions from 5 (such as we find some showstopper in TenFourFox 6 that requires an interim 5.x security release to be made). This will also snarl our home builders, so we may need to issue a workaround for you folks as well. We'll deal with this if action is required. As a point of completeness, 4.0.3 remains the terminal release for TenFourFox 4.

Speaking of 5, David Fang, our indefatigable Fink porter, reports that TenFourFox 5 now builds on 10.5 and 10.4 from the Fink unstable tree, and he and his crew are getting 10.5 support backported to TenFourFox 4. So those of you who want to continue making custom builds, but are allergic to MacPorts, continue to have that option. Strong work!

Finally, Chris Trusch's legwork on the Java and TenFourFox 5 issue has culminated in Apple essentially admitting that they broke Java on 10.5 on purpose in bug 668639 (except, of course, for Safari). The solution is, natch, to upgrade to Snow Leopard, and Apple says that the problem is a "non-issue for PPC users, because Firefox 5 and Chrome only run on Intel Macs." Don't we just love those guys? If we end up releasing a 5.0.1, we will have a workaround in it, or you can use the symlink fix detailed in the thread. The code for it will be in 6, but Java will also become unsupported, as I expect Update 10 to be the final Java update for PowerPC.

Watch for builds next week, hopefully. There will be a period of brief downtime on Floodgap while I physically move the hardware to a new network and new data center, but the Google Code page and this blog will obviously remain in operation. I don't expect the network downtime to be more than 12-24 hours.

5.0 and 4.0.3 finals

2011-06-21T05:42:00.000-07:00

With no major problems found in 5.0, the RC is now the final release. 4.0.3 is also released as a security and stability update only, with issue 63 also repaired, which marks the end of TenFourFox 4.

The next "beta train" for Firefox 6 leaves in two weeks. I'm in the process of moving to a new house, so this gives me a bit of a break before having to work on it. A "release pack" will be available with 5.0 changesets to apply to mozilla-release as soon as that repo stabilizes.

Unfortunately, Tobias' testing showed that tree vectorization didn't help much with gcc 4, which is still our baseline compiler. We'll look at this again when we consider jumping to 4.2 or 4.6. There are some other smaller things I'm considering for 6.0 that I can complete in the timeframe available. More on that later.

5.0 RC released, 4.0.3 plans, and onward to 10.4Fx 6

2011-06-16T21:42:00.000-07:00

For 5.0 we actually have a true RC, which I have released as (surprise surprise) version 5.0. If there are no major problems, then it will simply "become" the release version. The RC fixes issue 68, which turned out to be a JavaScript error Mozilla introduced that only affected us because we are pure tracejit (methodjit and tracejit together don't have the problem), and was fixed in bug 663789. It just so happened that pages that dynamically instantiated plugin elements were the most badly affected by it, but other pages could trigger it, and this doesn't change my intention to disable plugins by default in 6.0. However, no major issues have occurred with plugins otherwise, so 5.0 will be their last stand.

On or around June 21st, the following will happen:

The RC will become the final release, assuming a respin is not required. (If it is, you will be prompted to "upgrade to 5.0." Just do it, trademark of Nike, Inc.) General users will then be offered it.
4.0 users will be also offered 4.0.3. This is a security update for 4.0 and will be the last of the 4.0.x TenFourFox releases. It will also have the fix for issue 63. They can choose to stay with 4.0 if they have extensions that require it, or jump to 5. Note that we will be somewhat forcing people to stay in update order; i.e., if you skip 5, you will be still offered the update to 5, which you should do; then update to 6. The reason is to allow the browser to rebuild certain databases that may change from version to version.
Builders: new patches against mozilla-release instead of mozilla-beta will be available at that time as well. I expect little or no changes, but you should use those patch sets and that repo. If Mozilla offers source for Firefox 5, then we'll work up a wiki document about applying your changes to that, because mozilla-release will be unmaintained, used only for chemspills, and will be overwritten by the next version. If they do not keep a source tarball around -- which admittedly would not be SOP for them -- then we will have to and our tarball will have the patches already applied.

With regard to the security aspect, the rather spectacular Firefox WebGL video memory access exploit that has been making the rounds thanks to an (IMHO) rather impolitic and indiscreet disclosure does not affect us, because we disable it at the code level.

Onward to 6. Unlike the transition from 4 to 5, there is a lot more in the transition from 5 to 6 due to a full refresh cycle. However, we are less likely to have the initial panic I had trying to get 5 together because we are now compatible with the current Mozilla build system using Tobias' ported linker and I am not aware of any large changes in that, in IPC or in NSPR, so the odds of jumping to 6, while never guaranteed, look pretty good.

For 6, Tobias has investigated accelerating JavaScript by compiling it with vectorization. My initial attempts to vectorize the entire browser were shockingly unstable, but his initial tests with just NSPR and JS in issue 52 look promising and yielded around 10% improvement in benchmarks. We'll need to make sure that the same thing applies for stock gcc 4.0, however, and unfortunately our G3 friends won't get this benefit since it requires AltiVec. We will also want to bang on it thoroughly for the kinds of subtle errors such compilation can introduce. Once this stabilizes, and assuming it works, it's time to start thinking about porting methodjit to PPC and taking advantage of even more power, since IonMonkey is still a good couple of iterations away.

Plugins will be pref-ed off in 6 and there will be more switches for that so that you can at least still use them in 6, or use them in less unsafe situations, albeit without a safety net. By 7, however, I have a pretty strong premonition that they will break in major ways. We'll see. Even though we "have" IPC in 5.0, it is a gutted and almost completely non-functional IPC library in 10.4, present merely to make the build system happy and give it something to link against, and so plugins remain in-process.

Towards the end of a plugin-free existence is this remarkable proof of concept from the Mozilla team on implementing PDF viewing in JavaScript. Not merely a simple renderer, the code actually turns the PDF into JavaScript and executes it -- a genius idea, because the JS generated will also be accelerated, making PDF rendering remarkably fast. Although his demo has some obvious graphical glitches still, it is amazing how well it works even in this embryonic state. Look for it as an add-on coming soon.

Speaking of, a number of you had voiced concern about Mozilla moving too quickly with version numbers and leaving extensions behind (part of why I'm hedging our bets a little with 4.0.3). It does certainly seem like this is on Mozilla's mind as well, which is the apparent reason behind a meeting on risk mitigation they have scheduled before the formal Firefox 5 release. However, it appears clear that right or wrong, good or bad, rapid release is here to stay. Let's all curse Google Chrome together.

Release notes and builds for 5 RC:

G3
G4/7400
G4/7450
G5

TenFourFox 5 beta 2 released

2011-06-09T20:47:00.000-07:00

Beta 2 is released. This fixes the issue with widgets and bookmarks on Leopard (confirmed this time :), and has the video repaint fix I was waiting for. Sadly, this fix reduces speed a bit, but the display glitches the bug was causing were really quite unacceptable. People who also want to use the Ghostery add-on should see if the tweak in issue 67 fixed things for you.

Beta 2 also adds more AltiVec-accelerated scaling and YCbCr conversion routines to make three of four scalers converted, which reduce CPU usage while playing back video by about 33%. For example, my 1.33MHz iBook G4 can now play Mozilla's brand video with only occasional video freezes, and the 450MHz Sawtooth can now finally play it at all. In fact, VP8 (WebM) video due to our juicing and custom code now performs better than VP3 (Theora). The fourth scaler is a complex edge case and I'm trying to weigh how to best do it. I should also note that these AltiVec accelerated modules are actually only about half as effective as they could be -- since they are more or less simple-minded ports of the Intel MMX versions, they only use 64 bits of the 128 bit AltiVec vector registers because the MMX registers are only 64 bits wide. Once our versions are well-proven, I will try to refactor them to handle more data at a time; it's just easier to debug the algorithms if they are similar.

That's that for feature development in TenFourFox 5. There is one new bug that came out late while I was building beta 2 and this one worries me a bit, namely, issue 68. This has to do with plugins being dynamically instantiated by JavaScript, and it fails (and, possibly, crashes) on TenFourFox 5 for reasons I am so far unable to fully determine. I can wallpaper the issue, and wallpapering the issue will at least let us ship a somewhat "safely broken" plugin host in 5, but really, even if I do solve the actual underlying failure this absolutely must be the last version for plugins. There are too many ugly plugin-related bugs already with non-obvious causes and I can't wait to see how Fx6 and Fx7 will make them worse. The wallpaper fix is not in beta 2, but it will be in the next one if I don't figure out what caused it.

Read the release notes, then get:

G3
G4/7400
G4/7450
G5

5b2 delayed, sorry

2011-06-07T15:38:00.001-07:00

5b2 is being delayed a bit, for two reasons: 1) I want to get the complete AltiVec-enabled YCbCr-to-RGB conversion suite into it for testing (one part of four is in 5b1; I've finished a second part and am working on parts 3 and 4) and 2) bug 662450, which yours truly reported and is a Mozilla bug, but disproportionately affects us as we are unaccelerated. Mozilla is currently treating it as a blocker, so there should be some resolution to it soon.

I also landed issue 67, an issue with recursion in the JavaScript regular expression parser, which I'm a little concerned about functionally and don't want to land on 4.0.3 in case it has unusual side effects. Those of you who reported issues with Ghostery compatibility with TenFourFox 4 might want to try it with 5b2 when it comes out and see if that repairs your issues (Wolfgang O comes to mind, if you're reading).

As soon as the two issues are completed, I will run off new builds, which also fix issue 65 on Leopard. We will probably be able to get 5 final out on time with Firefox 5, and I plan to release 4.0.3 at the same time as the final release in the 10.4Fx 4 series. I have not seen any new landings on the Mozilla beta tree that would obviously break plugins, so assuming they still work, they will still stay. However, a big change landed on the tree for Firefox 7 that could potentially break them, so I still plan to pref them off in TenFourFox 6.

Beta Five is alive!

2011-06-02T20:47:00.000-07:00

Thanks to Tobias' linker and work on the build system, I am now pleased to unveil TenFourFox 5 beta 1 (roughly Firefox 5 beta 3, although it is hard to assign numbers due to Mozilla's new release strategy, so I'm arbitrarily calling ours "1"). And now you will get to play with it.

Firefox 5 is pretty much the 3.6 to Firefox 4's 3.5 (speaking of, 3.5 is toast, so if you don't want to use TenFourFox at least use 3.6 or Camino), which is to say, it is the minor feature and bug fix update. Actually, there are at least a few marquee features worth pointing out despite the brief gestation, and the big one is CSS animation, which is something WebKit has supported for awhile and now Mozilla's getting into the act. This is the technology that I personally hope will nuke Flash for good. You can see some demonstrations of it, including this simple animated text page and, impressively, the Mad Men title sequence rendered in CSS3 (high end G4 strongly advised). Most of the rest is ticky stuff and spec compliance. The return of WebSockets, a custom interface for permissions, and other bits of challenging electronics will not appear until Firefox 6, which will hit beta in about three weeks.

We haven't sat on our butt for this one, however. Besides the work expended on porting the Firefox/Chromium IPC code to 10.4 (mostly by gutting it) and hacking on the build system (more on that in a moment), we have some new features of our own. First, JavaScript acceleration is now enabled for browser chrome. This means that the browser itself, a fair bit of which is written in JavaScript too, now gets fed into our exclusive PowerPC JIT tracer. The improvement is not nearly as dramatic because a lot of features the browser handles do not benefit from it as much as web pages do, but it still makes the browser quite a bit smoother.

Second, JavaScript typed arrays are now enabled in the accelerator. These are a recent addition to Firefox allowing fast native storage of data, and are an important part of the audio API and certain high-performance applications (and WebGL, even though we don't support it). Previously TenFourFox could not use the JIT for this type of code, which meant such applications trying to do for speed actually ended up running slower. Now they don't. This means smoother audio and faster overall performance for these specific kinds of applications (note that this doesn't affect benchmarks much since SunSpider and Dromaeo don't really test this).

Third, we are starting to add more AltiVec acceleration to the content chain, as promised. First out is AltiVec-powered scaling, which uses a SIMD routine to speed up filtering and scaling of images and video. This is in this build already. The next step is to accelerate YCbCr ("YUV") colour conversion, and then specific optimizations for libjpeg-turbo (we are using the unaccelerated version in TenFourFox 5, but this is already quite a bit quicker than the old library, so it's already a win). This might slip to "TenFourFox 6," but hopefully some of it will appear in 5.

Other changes specific to TenFourFox include expanding the stack even further (now Kraken can run, which crashes 4.0.2; this fix will also be in 4.0.3) and dialing back the loop filter for WebM videos a bit, which gains some marginal improvement. Unfortunately, HTML5 video is still mostly the domain of the G5 right now.

Oh, yes, plugins: the "P" word. Surprisingly, they still appear to function, but I haven't exhaustively tested them. Since we have some higher priority work to do rather than expend time and effort into something I already know will be unbelievably unpopular, I'll let you play with it and if their support and operation seems no worse, I will leave them alone for 5. However, some changes are in the pipeline for 6, so expect it then. Please note that they do not run out-of-process; the IPC code is gutted and they can't even if they were capable of it. Also do note that by the time Firefox 6 (and hopefully TenFourFox 6) come out around August, almost a year of no updates to PPC Flash will have elapsed. Imagine how full of security holes and crash bugs it oughta be by then. Just saying.

Let's also talk about what had to be done to get it to compile (builders pay attention). Tobias' ported linker from Xcode 3 is now required to function and is available for download from the Google Code project. Although it appears to be perfectly fine to replace your Xcode 2.5 linker with (and supports 32 and 64 bit in one executable), I prefer to keep my old linker(s) around, so there is a shim that comes with the changesets to install in place of /usr/bin/ld which lets you dynamically change which linker you use (handy for making libtool use a linker other than the default). For details on that, see README.ld for how to install the shim and the Xcode 3 linker, and how to get the build system to use it. I am rewriting the build notes and those should be ready in the wiki already. You do not need to install the linker just to run TenFourFox, by the way, only to compile it from source.

The reason the custom linker is needed is because Mozilla requires everything linked together into a big orca-fat superlibrary called XUL. This library roughly consumes around 750MB of space, and the regular Tiger linker can't cope even in 64 bit mode. In debug mode it is not stripped and remains this size, but is runnable in Tiger. In the optimized builds, it is stripped down to the typical ~25-30MB size.

Part of XUL (a/k/a libxul) is the IPC system, derived from Chromium (yes, that Chromium). Most of it will not build on 10.4, but we are fortunate in that Mozilla doesn't use most of it -- they use NSPR. Therefore, getting it to build was simply stubbing out the functions that could not be converted until it finally compiled. The only part Mozilla seems to use is the IPC portion for out of process plugins, which we don't support anyway, but more to the point can't support: it requires posix_spawn, which does not exist in 10.4 and would be very hard to emulate. Fortunately Mozilla offers a way to blacklist plugins to always run in-process, and all plugins are blacklisted to do so in TenFourFox, so we don't ever run that code. We will have to revisit this problem when Mozilla goes to one-tab-per-process, however, because this code will be needed for it.

Things that need to be tested: I believe that our old update notification system should still work, but please let me know if you don't get update announcements. There will be one more beta prior to release, so you should get flagged when beta 2 arrives (don't be quick to update; wait and see if you get prompted). We will not support the auto-update strategy because frankly if you're using a Power Mac, I know you appreciate control over your setup and not being forcibly updated unless you explicitly ask. Also, like any beta, there may be Firefox-general bugs and you should be prepared for those, although I have been using TenFourFox 5 myself for the last week and it seems really quite solid.

Well, read the release notes, then go get it and tell me what you think:

G3
G4/7400
G4/7450
G5

Tobias gets a beer: Aurora on PowerPC

2011-05-29T18:37:00.000-07:00

Tobias gets a beer. This is the G5 build of the Firefox 5 beta, although for some reason it says Aurora. This was built with his 10.4-linked 64-bit ld, and appears to have all of our standard features. And it works!

~~Unfortunately this is not shippable either, as it is still building that monstrous 750MB+ libxul, so we're going to need to find some way of stripping that library down.~~ Fixed! Good ol' strip -x seems to have done the trick, and I will fix the build system to reenable stripping which was disabled for purposes of linker testing. Doing some more testing before issuing a formal beta. Meanwhile, I am considering options for 4.0.3.

4.0.2 final available

2011-05-28T16:08:00.000-07:00

While we sort out what we're going to do with Firefox 5, 4.0.2 final is now available. I have decided to abandon the base-maintenance distinction for patches while I wait to see how Mozilla will handle release driving for mozilla-2.0, so it will be the same bucket o'patches for right now.

Four-ever, yours

2011-05-26T18:37:00.000-07:00

Let's just come out and say it: after a number of sleepless days struggling with it, it does not look like there will be a "TenFourFox 5." Before you panic, that does not mean that TenFourFox will be unsupported or will not get new features! It just means we won't be building from the post-Mozilla 2.0 codebase anymore, and there is a bright silver lining to it.

Before we do that, let's post-mortem Firefox 5 on 10.4.11 PPC first. This is nerdy, so if you just want to skip on to the new revised roadmap, go for it.

As explained in previous entries, my main concern was that Mozilla as of Mozilla 5.0 requires Chromium IPC to be built (IPC is now required), and IPC requires building a giant library called libxul. In Mozilla 2.0/Firefox 4.0, this was optional, and we built without them because Chromium IPC requires lots of 10.5-only OS functions and libxul was too big to link with the Tiger 32-bit linker.

It is possible to get Chromium IPC to compile "just enough" to make Mozilla's IPC happy, because Mozilla's IPC code does not use very much of Chromium at all. Chromium includes lots of system abstraction code, which Mozilla doesn't need because it uses NSPR, and as a result almost all of the stuff that didn't build can be safely stubbed out with empty functions. The exception is the launch code, which requires posix_spawn and friends (which 10.4 doesn't have either), but even that right now is only used by out-of-process plugins which were marked for death in "10.4Fx 5" anyway. It would be a problem when Mozilla goes to full one-process-per-tab, but that is not imminent, and we can work around that too. So that much is doable, and does compile.

Linking, however, turned out to be hugely troublesome. Tiger has two linkers, the regular Darwin /usr/bin/ld which is full-featured but only 32-bit, and the 64-bit /usr/bin/ld64 which can deal with larger files but lacks many important features. In particular, a number of the indirect linking features Mozilla requires for dylibs are not supported in ld64, though they do give just warnings instead of errors. The reason this is important is that the linker can only deal with libraries that fit within its addressing space, and even with optimization and stripping libxul is too big to fit in 32-bit space.

The first problem was Apple's libtool: it doesn't let you choose your linker. To get around this, I wrote a shim Perl script to act as a switchable ld, calling ld32 (the renamed original /usr/bin/ld) ordinarily except if LD64 is set to one in the environment. I then hacked the Python build argument generators to call another Perl shim that looks to see if libxul is being linked or linked against, and turn on 64-bit linking if so. That much works.

The next problem, however, was that debug libxul can't be linked even by the 64-bit Tiger linker: it crashes with a malloc() error, strongly suggesting this is an operating system limitation. The size of this library, by the way, approaches 750 megabytes and I'm not even exaggerating. Since I doubt Mozilla would distribute an app that large :), something is already wrong.

But that wasn't all. For G4 and G5 optimized builds, the WebM accelerator assembly code requires special linker settings and ld64 doesn't support them either, so they couldn't be built as is. (ld32 couldn't even start, by the way.) Losing this code would be a serious blow to us, but would be doable if we could get the basic browser at least to stand up.

Unfortunately, we couldn't. The G3 version did, finally, appear to link in ld64 with a crapload of warnings, but when I started it up in the debugger it crashed immediately in darwin_gcc3_preregister_frame_info. This is a very low-level symbol in the C runtime (crt1.o) and suggests that dyld could not resolve the links. This could be for any number of reasons, including the app has to be in 64-bit mode for dyld to resolve references in a library of that size in 10.4 (a show-stopper), dyld can't handle a library of that size at all in 10.4 (also a show-stopper), the linker actually did fail and didn't realize it (also a show-stopper), or I munged the code while making it acceptable to the compiler to such an extent that it can't enter its main loop (and since we can't build the debug libxul, we have no good way of debugging it). It could even be all of the above.

Mind you, I'm not totally admitting defeat, but I am pretty confident now that we cannot build Firefox 5 as written on 10.4.11 PPC, at least not with our current tools. It should be possible to build it on 10.5.8 PPC, and there are some early builds on El Furbe of 4.2 (which was 5's original version number), but I don't know how well these work and his buildbot seems to have stopped again. That doesn't help us, though, because we support G3 Macs and 10.4, and we always will. More to the point, I personally only run 10.4 and can't support a "TenFiveFox." If someone out there wants to apply the TenFourFox enhancements and run off 10.5-only builds, I will gladly direct people their way, but here we stay on 10.4.

I know Tobias and some others (David Fang?) had built some custom tools themselves. I am doubtful that the tools are the actual problem (I suspect the real issue is with 10.4's dyld), though I can't test that myself because I don't want to corrupt my working build system with unofficial tools, but here's your chance to put your money where your mouth is. If you can get it to link with your custom linkers, you will have done us all a great favour and win our userbase's ever-lasting regard, plus proven that they work. Remember, it's got to work in 10.4 to qualify, but as long as the browser can start, there is hope. Here's how:

Make sure your system is set up to build the current TenFourFox as a prerequisite (see the wiki) and that it can, in fact, build TenFourFox 4.0.
Clone the mozilla-beta repo.
Download the current set of Firefox 5 patches and serially, in numerical order, hg import them into the repo.
Move your /usr/bin/ld to /usr/bin/ld32, and move SHIM.ld in the repo root to /usr/bin/ld (you need to do this as root, obviously). You don't need to undo this at the end unless you want to as it is designed to be transparent. Replace this with your custom linkers as indicated, of course.
Build as instructed in the wiki. Good luck. Post your experiences.

Welcome back, non-techies, now to the roadmap.

When Firefox 5 comes out, it will move to mozilla-release, and then in 6 weeks be obliterated -- no more numerical branches. At that point, assuming no one has cracked the problem we discussed above, we drop source parity and continue our development purely from Mozilla 2.0 at feature parity state.

First, let's discuss what this means practically. Camino is very popular with PowerPC users, and 2.0 is based on Firefox 3.0 (Mozilla 1.9), which is three full generations behind TenFourFox. Despite this, it still manages to get the job done for a great many things; I myself am a former Camino user. Even Camino 2.1 will "only" be based on Firefox 3.6 (1.9.2). So even if we just phoned it in and did only security updates, functionally speaking TenFourFox would still be generationally ahead for several years. Remember how long WaMCom usefully persisted on OS 9 with nobody at all updating it?

But, of course, we will do feature updates (since we will be at feature parity), and because we are very close to Firefox 5 we can adopt quite a bit of the source code with only minimal changes and risk. We do need to pick features and support them with fixes, but the process should be much cleaner and less iffy than the case with Classilla where perilous backport glue code is sometimes necessary (and not always successful). We cannot take code that depends on IPC, of course, but layout, content and media should work, and some careful graphics updates. However, we should not mess with the UI except if it's buggy, because this may require saddling our slower users with more baggage than their computer can handle and the current browser at least is a known quantity. We will also wait on JavaScript updates in case they break the nanojit, and IonMonkey is in the future, so we should see what turns out with that.

And, since we aren't moving to TenFourFox 5, updates will still function properly in the way they are, and the plugin code won't change. So you get to keep plugins after all. I hope you're happy. ;) My arguments over security still stand, however, and I still strongly recommend something like Flashblock.

This is the new feature parity roadmap; my intention is to have a release every six to eight weeks, with four to six weeks of dev time and two weeks of "pre" testing:

4.0.2 still comes out probably this weekend. It looks very successful; no major problems were reported with the second beta, and the JavaScript speed and stability increase was universal.

4.0.3 is the next feature release. This will turn on the nanojit for the browser chrome and hopefully add load-store to the nanojit (these were features intended for TenFourFox 5 originally), along with configurable WebM filtering settings. It will also include security fixes and Firefox 5 bug fixes that we can safely backport. It may or may not include some new safe-looking features that do not significantly change the browser, such as the Firefox 5 changes and enhancements to HTTP and XMLHttpRequest. I might defer these to a '4.0.4'.

4.1 is where the magic will start. Among the features I think we need to have include Firefox 5's improvements to canvas, CSS animation, WebSockets (when they finish screwing around with the IETF spec, likely by Firefox 6) and SSL False Start. These are all features likely to be leveraged by future sites. Similarly, we should try to adopt some of their SSE/NEON vectorized improvements, especially for text scanning, along with libjpeg-turbo. These features are developing and we need to constantly watch for new bugs, so we should keep this set of new supported web features small and essential. If 4.1 is delayed, there will be a 4.0.4.

Because 4.1 changes the browser core in significant ways, we should not report ourselves as Firefox 4 anymore. We might adopt a built-in user agent changer like Classilla. More on that later.

So, the road moves on, just in a different direction. I'm going to start writing up new roadmaps and worklists in the meantime while 4.0.2 builds on the G5. And for the hacker builders, you've got eight weeks or so of Firefox 5 to see if you can get it running and succeed where I failed. In the meantime, TenFourFox remains "four-ever, yours."

4.0.2pre beta 2, crushing benchmarks, changing lives, etc.

2011-05-18T20:38:00.000-07:00

I bet chtrusch is pretty proud of himself for ruining beta 1 with his bug report, and I hope he's happy. Actually, I know he already is, and you'll be happy too, because with the test case he reported we have finally stamped out issue 37 which was a long-running JavaScript crash bug we had wallpapered over for 4.0 and 4.0.1.

And, because that wallpapering was done by blacklisting certain traces in the JavaScript accelerator, we no longer need the wallpaper since the underlying problem (running out of stack space) has now been fixed by giving the app more stack. SunSpider was a mixed bag; the G5 barely moved, but the 7400 had an unpredictable improvement of around 5-15%. However, in V8, the quad G5 in Highest went from a score of 656 to a reliable 822 (higher is better), and Dromaeo -- which includes V8 in its composite score -- also duly increased, from 93 runs/sec to 110 runs/sec!

While you're testing things, please also look at the good work Narcotix and chtrusch are doing with localizations in issue 42 and issue 61, including a locale installer so that you don't have to manually sling files around.

Before we get to the goodies, some business items. I'm trying to work in with Mozilla about security content and that means being a good neighbour about currently extant security issues. Mozilla still does not plan to issue a formal Firefox 4.0.2, but they are amenable to putting fixes into the mozilla-2.0 tree for downstream consumers such as ourselves. They do not, however, want such fixes distributed outside of the Mozilla repos until Firefox 5 is released, so we probably cannot release the maintenance changeset pack until then as this contains the actual fixes involved. Sorry, builders. The current release date for Fx5 is still June 21st, although it may slip due to the rapid release process not yet having been tested for full rollout. However, I am asking explicitly to make sure we can still get binaries out to your good selves on time June 1st.

There are a couple other potential security issues I am reviewing for inclusion, though I may defer these to 4.0.3 pending Firefox 5.

Speaking of, Mozilla is no longer planning to issue numbered release branches with Fx5 - i. o. w., there is going to be only one mozilla-release repo and it will only contain the current version; there will be no archival branches like they used to have in the CVS days and in hg until Mozilla 2.0. This is a problem for our builders because when the next version comes down the pike, there will be nothing to pull from and apply changesets to. What we might have to do is clone and hg export a tarball so that the builders still have a stable "source source" while we retain source parity. Hopefully, however, Mozilla will keep stashing their own tarballs somewhere and we can build on top of that, perhaps. I'm still not sure what to do with this yet and we'll worry about it if we get Fx5 running.

In the meantime, change your life today with beta 2 (and crush benchmarks, etc.); read the release notes first:

G3
G4/7400
G4/7450
G5

4.0.2pre now available

2011-05-15T18:15:00.000-07:00

4.0.2pre "beta" is now available for testing. The only TenFourFox specific change is issue 58, suggested by chtrusch for those systems that don't react well to plugins forcing smooth scrolling; you can now set tenfourfox.layout.smooth_scroll_plugins to false if you don't want that to happen. Speaking of plugin prefs, there are now separate prefs for full-page and embedded plugins, and actually now would be a good time to open about:config and type tenfourfox in the filter box to see all the custom prefs that are available presently. There will be more in the future.

The compiler changes we were hoping to take advantage of unfortunately made the browser dreadfully unstable -- it couldn't even open a window. We will not mess around with the compiler again until we must use gcc 4.2.

As mentioned previously, there will be no official Firefox 4.0.2 -- TenFourFox 4.0.2 is the same as Firefox 4.0.1, with security patches I have reviewed and applied while we wait for Firefox 5 to enter the beta channel and I can investigate if I can port it. This is part of our commitment to maintain a secure browser regardless of what actual version of Mozilla/Gecko we are using. There should be no compatibility issues. You can see the security issues applied (please note that some will require security privileges to view) by reading the rough draft of the release notes. On final release, these will be in the maintenance changesets, minus test cases so that we don't leak proof-of-concepts for any bugs that are still sec-locked. The patches in question were already approved for Firefox 5 and we are simply backporting them to Firefox 4.

Firefox 5 goes to beta channel tomorrow and I plan to pull source and do some preliminary work on it in between moving into my new house. For now, ~~please try 4.0.2pre.~~ UPDATE 5/17: See comments; a crash bug has been isolated and repaired. Stand by for a rebuild tonight or tomorrow. If there are no issues, it will be converted into final release on June 1st.

G3
G4/~~7400~~
G4/~~7450~~
G5

Watching the floor drop out under our feet

2011-05-11T16:13:00.000-07:00

Well, this was unexpected. Multiple places across Bugzilla with 4.0-ready patches are now being denied to land on mozilla-2.0, and the reason appears to be that there will not be a Firefox 4.0.2. Instead, Mozilla is dropping Firefox 4 after 4.0.1 and going straight to Firefox 5. In fact, Firefox 3.6 will be supported longer than 4.0 will!

(I should clarify that while a dummy 4.0.2 release is being [lightly] considered as a migration strategy for the end of Firefox 3.5, it would have no new content and would explicitly be just a version bump. We aren't going to do that.)

That was the bad news, now for the good news. My request to receive Mozilla security group privileges was accepted, and I am now able to view security issues as they are submitted and as they develop. Using this new access, I've been able to identify some security and stability fixes that apply to 4.0.1 (and some to 3.6, but there will be a 3.6.18 for those of you who are still unsure about jumping to TenFourFox) and that can be accepted by us safely, to keep us at security parity while I ponder what to do with Firefox 5 as it enters the beta channel.

Because this essentially forces us into unsupported mode early, we're going to adopt a new strategy to reduce branch-to-branch-jump work and yet crosspatch future patches to our codebase in a reasonably straightforward manner. Once we lose source parity, this will change, but as long as we maintain source parity and are built from a Mozilla release we will maintain this new strategy (builders take note):

- Each branch will have a base set of patches specific to that branch. In the case of Firefox 4.0.1, it is the changeset that is already available and that you can download now. These sets are the "base" and are the unique TenFourFox portions the browser is built upon. They should be unique to that branch and should not carry future work unless absolutely necessary, i.e., no changesets that would be part of a future Firefox branch. The reason for this is that the base will be carried forward to the next release of Firefox and modified as needed, so we don't want changesets that would already be in that release (most importantly stability and security fixes we backported from the future branch). The base may be added to if we find TenFourFox-specific bugs that we need to repair in point releases.

- On top of that will come specific future changesets that we are backporting for stability or security. There are ten of these I am reviewing for probable inclusion in TenFourFox 4.0.2, and about six others of low severity that we may or may not take. None of these are highly critical, but we should fix them. These changests are already in Firefox 5; we do not want to carry them forward in the base. These changesets will be maintained separately and will not be reapplied to "mozilla-5.0" or whatever they're going to call it because they don't need to be. The version and milestone should be maintained here too because we definitely don't want to have to merge that in base.

So, for 4.0.2, new base changes will be ~~the compiler string modifications and~~ UPDATE: the compiler changes make the app really unstable, so we're not shipping them for 4.0.2 the pref for enabling or disabling smooth scroll with plugins. We will then establish a maintenance changeset with the new version and milestone numbers and the security/stability patches, which I'll document in the release notes. Builders need to apply both the base and the maintenance changesets to generate a complete version. When Firefox 5 comes out of beta, then we'll immediately clone that repo and apply the Firefox 4 base to it, making any additional changes that we will need and thus creating a new base for that release branch (assuming it works, of course -- see the previous entry for why this is probable but not guaranteed). Assuming a TenFourFox 5, while we wait for Firefox 6, we will watch for security updates and backport to 5.0.1 as necessary.

I have not yet decided if there will be a 4.0.3. If there are no major issues by the time TenFourFox 5 dawns and it seems to work, then we will simply release. If there are some issues identified of sufficient severity that do still affect us, I will issue a 4.0.3 for those who can't give up plugins just yet so that it can buy them some time.

Like every version of TenFourFox, you will never be upgraded automatically to a new version unless you download and install it yourself. I don't like this personally and I don't like forcing it on people, and we don't have the infrastructure for it anyway. If we liked forced upgrades, we wouldn't be using Power Macs. ;)

The current plan is to select and land patches this week and chug out builds over the weekend for you, the loyal beta audience. 4.0.2pre should be available on Monday, and if there are no major issues 4.0.2 final will come out on June 1st. As always, comments and questions solicited.

Towards TenFourFox 5 and 4.0.2pre

2011-04-25T14:46:00.000-07:00

I'm putting this post out a little early to talk about what we're going to do with Firefox 5 because my hand's been forced a little bit; I've been seeing some well-intentioned blog posters saying, among other things, that TenFourFox is nice and holds the fort but isn't going to get any updates. Besides the fact we just had one, this is unequivocally, unbelievably, unimaginably false. I personally use TenFourFox; I wouldn't shoot myself in the foot by failing even to maintain security updates, let alone additional features. When (not if) we reach a state where we cannot hack "Firefox.next" to build on 10.4 PPC, then we simply carry on maintenance on the branch we know is still working. If nothing else, the browser will remain safe to use even if the technology reaches a state of arrested development. Don't forget that Camino currently uses a three-year-old renderer, but they maintain and land security fixes from later branches to keep the browser safe to use. We'll be doing the exact same thing. I'll discuss that in more detail in a moment.

On that note, let's talk about Firefox 5. Firefox 5 really should be called 4.1 because it is, indeed, the minor feature update to Firefox 4 (as opposed to 4.0.1, which is the minor maintenance update). That said, it does have a number of important features we will want. Very few of these are user-facing, mind you. I've been playing with Aurora (essentially the "stable alpha") on my lonely Core 2 Duo mini, and at a UI level you're not going to see much difference. Like Snow Leopard to Mozilla's "Leopard," Firefox 5 really improves very little of the interface relative to Fx4.

Of the features that are there, however, the most important one is CSS Animation. Remember, Flash or any other plugin will not be enabled in "TenFourFox 5," and in this HTML5 world, we want to encourage animation and graphics methods that do not rely on Flash. CSS Animation is part of that transition away from proprietary, buggy and (in our case) unmaintained ways of doing graphics and animation in the browser. There are also quite a few performance improvements in Firefox 5 that will surely benefit us, because sadly our machines aren't going to get any faster by themselves.

For TenFourFox 5, in addition to getting all that working, I would also like to expand some of our marquee features to add even better performance. For example, the JavaScript nanojit is now stable and has achieved enough cross-processor parity that we can flip it on for the browser as well. In fact, I've been running my 4.0.1 in that manner for over a month (go into about:config, set javascript.options.tracejit.chrome to true), which has been nice and stable, and gives a noticible speed boost at the cost of additional memory needed for the traces. This is kind of a big change to throw at 4.x, but we definitely want it in 5.x. I'm also hoping to finally fix once and for all the problem with our generated prologue and certain kinds of native code calls, which are handled currently with a safe but slower abort to the interpreter. If these calls can be embedded in the trace, we can get even faster, but this has an obvious stability impact and I want extensive beta coverage. Finally, I want to support JavaScript typed arrays in the nanojit; these currently fall back on the interpreter, and the apps that are likely to attempt using typed arrays for increased performance are ironically going to perform worse on TenFourFox because of the fallback. This work has already been done preliminarily. I think this work list is fully achievable, especially because a lot of it is already done or in progress.

In the would-be-nice department, I want to continue our progress towards full AltiVec acceleration of the content chain. We've started (I think very successfully) by enabling AltiVec in libpixman and libvpx. There are some trivial additional code conversions that I can do in libvpx, and I'd like to try implementing them. In addition, Fx5 adds libjpeg-turbo, which is already faster in and of itself, but also gives us the chance to convert the MMX/SSE code it includes into AltiVec/VMX. This should greatly speed up our image display and processing. Finally, I want to track down other places in the code where Mozilla has already implemented SSE or SIMD equivalents for C code and write VMX versions. This work is unlikely to be fully completed for TenFourFox 5, but some parts should make it.

As I alluded to previously, Firefox 5 is not a slam dunk: we need to get Chromium IPC working on Tiger. I know it's possible to get it to compile, but linking was historically an issue because with IPC everything is clotted into a huge superlibrary called libxul (and in debug mode, this was too large for Tiger's linker to pull together), so it has never been tested. Fx4 does not require it (just strongly encourages it), but Fx5 demands it. Again, I remain cautiously confident the port is possible, but even if it is not, let me reiterate again that even if we remain stuck on Mozilla 2.0 forever, we will still get security updates landed on TenFourFox and the browser will remain functional and safe. You have my word on it: I eat my own dogfood. I hope that Classilla's longevity and update history demonstrates to people that TenFourFox is here for the long haul too.

That aside, Fx5 sounds pretty good, right? Well, don't get too raring to go just yet, because we have to consider the ramifications of the migration. Since our last discussion on this topic, more information has become available, and most of that information from our perspective is bad news. Fx4 will be the last of the "big" releases and the last to retain the old release EOL timeframe, which is usually six months of updates after release of the next version (and you can bet that under the new rapid release framework it won't be a second longer). Fx5 and later will be unsupported the second the next version comes out.

What does that mean for support? Well, barring something extraordinary, as soon as Fx6 emerges there will be no more Fx5 updates. Period. Similarly, when Fx7 comes out (!), no more Fx6. This means the safety net that would ordinarily be under us with a stable branch like Fx4 will no longer exist once we jump to Fx5. If there is a serious bug discovered in Fx5 and Fx6 has already come out, then there will be no fix for it in Fx5; Mozilla wants you to simply update. This is not ill-conceived on their part, but it sucks for people like us trying to actually build around the updates because the ground keeps shifting under our feet, and may require us to do a significant amount of backporting. I'm also not sure what this will mean for extension compatibility. Mozilla has plans in place for carrying forward addons semi-automatically, but if this process cuts off backward compatibility for old versions at the same time (as, possibly, a carrot for updating), that could really bite us.

To balance this and to spare much unnecessary work, this is how the rollout will happen, assuming the port of Fx5 is successful.

- Work on TenFourFox 5 will not commence until Firefox 5 is released. Other than chemspill releases, we know no more work will occur on the final release branch once it becomes final because the next 6-week release will clobber it. Therefore, we know nothing is going to change for Mozilla 5, and this simplifies our porting.

- Until work begins, there will be 4.x updates, and at least one more 4.x release (4.0.2pre; we'll get to that) will have feature work.

- Once work begins, we will have our own separate formal betas, just as before. I envision no more than three, most likely. 4.x updates will continue simultaneously as pure security and stability updates only.

- Once we are final, we will publish a limited set of 5.x security and stability releases while we look at Firefox 6, which should be out by then or soon. I am working on getting security access at Mozilla so that I can review and backport security fixes as they happen, though in the short term we can still get this information from publicly-available Mercurial commits, and they will land on 5.x too.

The cycle will then repeat as long as we are able to compile Firefox.next, whatever it happens to be. If we jumped to Firefox 6, then there will be 6.x betas (with 5.x releases), followed by 6.x releases while we look at Fx7, and so forth. In essence we will be emulating the old Mozilla release mechanism, except that we will be actually maintaining our own old branches instead of relying on Mozilla to do it for us.

At the point where we can no longer move forward, we set up our own Mercurial repo and maintain the last good branch "forever," backporting all security fixes that apply, any bug fixes that apply, and (very carefully) new features that we might need -- limited largely to browser core, layout, content, DOM and JS. JavaScript is pretty straightforward to keep current, fortunately, because it is largely self-contained. The rest will be examined on a case-by-case basis. This is what I've referred to as security parity and feature parity in the Wiki.

I really hope this puts people's concerns to rest; TenFourFox is here to stay. I need a browser that works on my G5. You can rely on this because I have my own skin in the game, as do, presumably, others who have contributed and who will contribute in the future.

Now, about 4.0.2pre. This is a feature release, but with only one feature, which is to try new compiler settings. Since this can lead to very subtle bugs, I really want to keep everything else constant to avoid nasty little variables I can't control for. Part of this will be to enable -ftree-vectorize for G4/G5 and attempt a proper 32-bit compile for G5 without using the hackish hybrid compile string we use now (64-bit Firefox on G5 still has issues and we don't really benefit from 64-bit anyway in the same way that x86 does). This won't affect the G3 releases, obviously. Similarly, it will include whatever other fixes for Firefox 4.0.2 that Mozilla has landed. By the way, they are holding their official release of Firefox 4.0.1 until later this week due to a last-minute issue, but this is apparently an oddiment of their update infrastructure and does not concern us. 4.0.2pre will be available when Mozilla releases it to the beta channel and I'll give you a heads up when it is about to arrive.

Well, that's enough blather; if this does not put your questions to rest, ask them in the comments.

4.0.1 final is pushed

2011-04-22T18:25:00.001-07:00

4.0.1 final is pushed to Google Code and update notifications should be going out. Those of you on the beta edge shouldn't notice anything different; other than a quick internal branding change, it is otherwise the same as 4.0.1pre. I'll have more to say about 4.0.2pre and 5 in an upcoming post. For now, grab it and carry on.

4.0.1pre is now available

2011-04-15T22:52:00.001-07:00

So, here's 4.0.1pre, and here's what's in it:

- The revised G5 JavaScript nanojit. Which is, you know, faster. And therefore doesn't suck. (G5 only)

- The AltiVec accelerated WebM video decoder (VP8, to be more precise). (7400/7450/G5)

- The scroll-slower-upper-thinger to make Flash applets not quite artifact as badly. They will still smear with scrolling, but less noticeably. (All)

- All the Firefox 4.0.1 fixes to date. There are quite a few; some don't affect us, but there are some security-related issues in here (none public) and some crash fixes. (All)

Plus this bonus hotness I snuck in:

- Tuned movie playback for all platforms, which affects not just WebM but also Theora. This causes slower Macs to buffer more decoded frames and smooth out the audio even if the video is choppy. The settings are different for G5 than G4/G3 because G5 has much bigger bandwidth due to its wider front-side bus. (All)

- Enabled AltiVec compositing also for libpixman. This means that pixel compositing is now done with AltiVec operations, which affects just about everything in the browser, really. The effect is subtle, but smooths out animations nicely and improves repainting speed. (7400/7450/G5)

Before you download this, consider grabbing a video and watching it in 4.0s just to see your starting point. I recommend going to www.youtube.com/html5 and signing up for the WebM trial (you don't need to have an account for that, just join with the link at the bottom), then viewing this video about Google's fiber experiment. If that video comes up in Flash, you did it wrong. This is a good test video because it has scenes with high data rates (the product manager on camera) alternating with lower ones (the animation sequences in the middle, and the title card and end card), so it's a good general overview.

On my G4/450 Sawtooth (7400) -- and by the way, I still don't support systems slower than 1.25GHz for video -- this video is pretty much unplayable with 4.0s. The audio is immensely fractured and you can forget about any video frames.

On my iBook G4/1.33 (7450) with 4.0s, the video and audio are extremely choppy with Energy Saver set to Automatic. Some frames appear and the audio is at least intelligible, but still fractured. It improves marginally at Highest.

On my quad G5 in Reduced with 4.0s, it plays in fits and starts. The snippets it does play are normal, but its data pipeline poops out and it has to buffer again repeatedly between them. On my quad G5 in Highest, it plays with only rare audio artifacts. I don't have a G3 handy right now (my PDQ is in pieces on my workbench waiting for a new hard disk).

So, now install 4.0.1pre and play it again. On the Sawtooth G4, the audio is artifacted, but now intelligible. The video is a glorified slideshow, but frames do appear at least occasionally. Hey, WebM video is pretty hefty to decode, and this computer is eleven years old, so whaddya want? :P

On my iBook G4 in Automatic with 4.0.1pre, it skips frames frequently, but the audio is nearly intact. In Highest, it still skips, but fewer, so I'd consider this playable.

On my quad G5 in Reduced with 4.0.1pre, it now plays perfectly.

Please note that Mozilla's streaming code is not terrifically robust. For example, if you enlarge or contract the Google video gadget using the arrows icon, then you start getting video artifacts when something overlays the video, and the buffering starts to seize up. The code doesn't seem to be able to handle a sudden drop in throughput well (a similar effect occurs when you change processor speed midstream in System Preferences). You can fix this by rewinding back to the beginning of the video, and then the streamer will be able to properly buffer. It looks like Mozilla's code just doesn't know what to do with the CPU changing performance characteristics abruptly or the video being dynamically resized as the video plays, but in fairness this is probably a lot to ask of it.

By the way, only my quad G5 in Highest could handle enlarged video fully (in Reduced, it got choppy), and don't even think about playing it full screen, because we can't hardware blit (this poor G5 got absolutely crushed trying to play Big Buck Bunny at 1920x1080 in software). This will only get better and still has some getting better to do, but it's a start, and the combination of buffer adjustments and VP8 VMX decoding does help.

Note that the WebM container decoding is still in C, and the actual portion of Mozilla's code that does the conversion to pixels (before they are blitted, which is VMX-accelerated) is also "just" in C. Firefox 5 is adding SIMD-based decoding of JPEG images; in at least older versions of libogg there was some AltiVec support; and I'd like to at some point have an entire AltiVec-accelerated content chain, all the way from raw data to rendering. There is some code that I think may be trivially converted to AltiVec (either in C or assembly) to gain us even more speed on the non-optimized sections, but that's for a later time.

Here is what I need from you wonderful beta hounds:

- G3 owners: Expect scrolling to be slower when a Flash applet is on-screen, but is it too slow? WebM video probably won't play hardly at all on your system, but it should not crash (which would mean AltiVec code snuck in). Is there any playback degradation with Ogg and VP3/Theora video using the new buffer settings?

- G4 owners: Improvement noticible? Above 1.25GHz, is video at least acceptable, even if it's a bit choppy or imperfect visually?

- G5 owners: Improvement noticible? How is JavaScript performance now?

Mozilla is planning an April 26th release for 4.0.1, and so will we. Anyway, go get:

G3
G4/7400
G4/7450
G5

You should get an upgrade notification when 4.0.1 final is available, so grab it and have at it, and post your observations in the comments.

I am the world's biggest liar

2011-04-12T20:48:00.001-07:00

Dear readers, I must confess to you all what an amazingly brazen and horrible liar I have been. For days, nay, weeks, you have laboured under the completely fraudulent impression that AltiVec-accelerated WebM video was coming to 4.0.2pre, and my conscience, ravaged by guilt and dismay, cannot abide to persist in my duplicity any longer. I must therefore reveal to you the awful, shameful truth ...

... that AltiVec WebM is going to be in 4.0.1pre! YAY!!!

Yes, it actually works! With a little preprocessing of the included AltiVec sources from Google libvpx, some adjustments by hand and a lot of glue code in the build system, we are now building a mostly (not fully, I'll explain in a second) VMX/AltiVec-accelerated VP8 codec, just like the SSE2 and NEON-juiced VP8 codecs for x86 and ARM! It'll be in the 7400, 7450 and G5 releases.

This version now makes all but the highest data rate videos at least playable (and some completely playable) on G5 and high-end G4 machines, and makes it possible for video to play at all on low-end G4s (but please note that the recommended 1.25GHz clock speed remains). As my standard, I used a clip from Big Buck Bunny which would only play fully on my quad G5 if I turned Energy Saver to Highest (I usually run in Reduced to save power and increase the life of the machine), otherwise the data pipeline would run dry and it would seize up repeatedly. With the new AltiVec VP8, it runs all the way through. No stutters, no hiccups. YouTube HTML5 performed splendidly. It's a beautiful thing.

Oh, but that's not all that'll be in 4.0.1pre. Besides the G5-enabled JavaScript acceleration (down to 1760ms in SunSpider!) I talked about in our last post, I've also found a kludge to reduce Flash screen artifacts when scrolling. I'm sure the careful ones out there have noticed that scrollllling verrrrry sloooooowllly will keep the artifacting down to a minimum, and there's a way you can do this already: enable Smooth Scrolling under Preferences, Advanced, General. However, smooth scrolling is definitely slower and it stinks to do it all the time if you're not used to it. So, why not enable smooth scrolling when a plugin is onscreen, and then revert to the user's preference otherwise? Why not indeed! And, while there is still some artifacting, it is much, much less. Of course, if you use HTML5 video like our new AltiVec WebM, you won't need Flash. I'm just saying.

This is not an unqualified success, however. I said the WebM code is mostly AltiVec accelerated, and it is. However, we do not have assembly source for most of the inverse discrete cosine transform algorithms, just for one of them. I looked at the C version for the other inverse DCTs and it looks pretty obvious to vectorize, but this is going off into completely new work territory and I think I'd rather not do that for a stable branch even though this is getting beta coverage (read on). Fortunately, the especially computationally intensive parts such as the filtering are fully written in assembly and we do have those. Also, this means we have points to improve on in the future, so performance should only get better.

Also, just because the decoder is AltiVec-enabled doesn't mean the compositor is, and I've alluded before that the graphics stack in Firefox 4 is slower than Firefox 3.6 on non-accelerated systems (and all TenFourFox builds are non-accelerated because PPC Tiger lacks OpenGL 2). If you try to play a WebM video expanded or full screen, then you're also testing how well we blit to the screen and scale the image, and we already know that's a bottleneck. At least for now, full screen video will still be the domain of Flash Player. (I was hoping Cairo 1.10 would land in Firefox 5, but it looks like it won't make the cutoff after all.)

Do note that the reason I want to put all this hotness into a quickie beta release is because this is a lot of new and relatively untested code. G3 owners are particularly important because 1) I don't want AltiVec code leaking into your builds and crashing you and 2) I want to make sure that the plugin scrolling hack doesn't make your machines in particular too slow. (Flash itself might, ha ha, but we shouldn't.) Similarly, I want to make sure that the AltiVec acceleration on G4/G5 is as good on as wide a range of systems as I think it is, and ditto for JavaScript on the G5.

This is all very convenient because Mozilla is planning to release 4.0.1, which they have named "Macaw" (presumably after watching Rio trailers and playing a lot of Angry Birds), on March 26th. With luck, there should be nothing landing on the 2.0 release branch tomorrow, so I can pull down the changes, spin off a few builds, and hopefully have betas out to your lucky devils likely by this Thursday or Friday. There are a lot of fixes in 4.0.1, mostly for crashes and a couple for some possible security issues. The plan is to release our own betas and assuming they pass muster, we release the same day as the regular 4.0.1 to the general audience. In the meantime, I'll have more to say about Firefox 5 in a future post.

Anyway, will you forgive me for my lies and heartbreak? I knew you would.

Attention G5 owners: your JavaScript no longer sucks*

2011-04-08T14:54:00.000-07:00

(* or, what Apple never told developers about the PowerPC 970)

It's time for celebration. My quad 2.5GHz G5 is now benching 1760ms on SunSpider using the internal 4.0.1pre I've whipped up, and a whopping 93 runs/sec on Dromaeo. That's a little over half the raw interpreter's SunSpider runtime (about 3370ms), and over half of the limping 3710ms it gets with the current hybrid interpreter-nanojit used by the G5 build of 4.0s. I bet you lucky dogs with dual 2.7GHz G5 systems will see even better. Everything is faster. Birds are chirping. The river is high. Gold bricks are falling from the sky, and not on people. This is so tasty it will be in 4.0.1, as so far it's just as stable as the G3 and G4 versions and runs even faster.

Since this blog is designed around some technical nerdosity, let's engage in a little, and also collect what I've learned over the difficult process of getting the nanojit tuned for the G5.

First, a little history for people who don't know what the nanojit is. Mozilla has three layers of JavaScript: the base interpreter (SpiderMonkey), the nanojit (TraceMonkey) and the methodjit (JaegerMonkey). The base interpreter is the base interpreter: it's pretty good as interpreters go, but it's just interpreting code and does not compile it. Until Firefox 3.5, this was the only way JavaScript was run in Mozilla-based browsers. On Power Macs, 3.5 and 3.6 still use it, as do Camino and SeaMonkey, and so do all the community PowerPC builds of Firefox 4 other than us.

Mozilla had been looking at a means of compiling JavaScript, and collaborated with Adobe on what was then called Tamarin, a new JavaScript engine. Tamarin overall actually turned out to be slower for the purpose, but a lasting advance from the Tamarin project was the concept of the nanojit: a "tracing just-in-time compiler" that watched for frequently executed portions of code, then recorded the atomic operations being generated in a special intermediate language called LIR, and finally compiled the LIR into machine code. This nanojit-based accelerator was christened TraceMonkey (no, I don't know why they have such a simian fetish either). Versions were immediately made for x86, and later other architectures followed, including ARM, SH4, SPARC and even MIPS. Adobe wrote up one for the PowerPC too, but it was not complete enough to be used in Firefox, and this was not rectified in Firefox 3.6. That's why all official Mac PowerPC builds of Firefox have such comparatively slow JavaScript performance.

(In Firefox 4, Mozilla added a more conventional method-oriented compiler, based in part on Apple's Nitro JIT used in Safari. Since Firefox 4 is Intel and ARM only officially, the methodjit only runs on x86, x86_64 and ARM. This combination of the nanojit and methodjit is JaegerMonkey. There is no PowerPC methodjit.)

For TenFourFox beta 9, then, we took that partially written PowerPC nanojit and finished it up, thus being the first PowerPC-based Mozilla browser to implement TraceMonkey. My first tests were on my G5, which were very disappointing. Although many operations were significantly faster, many operations were significantly slower, and I concluded that the nanojit didn't seem to be a good fit for the PowerPC ... until alert users tested it themselves and told me it was ridiculously fast on the G3 and G4. To get around this, I compulsively benchmarked various JavaScript low-level operations on the G5 and cut out the ones that seemed to be slow. This was good enough to beat the baseline score for Dromaeo and V8, and not suck too badly at SunSpider, and this got released in beta 11. At least at that time, that seemed about as good as it was going to get for the PowerPC 970.

Earlier this week, I was busy working on enabling VMX (AltiVec)-based WebM decoding (which by the way is about halfway done -- I now have it integrated into the build system and have started converting the existing assembly language code to the dain-bramaged as assembler Apple stuck in Tiger) and while investigating an oddiment in the assembler's syntax I stumbled across a key note I hadn't seen mentioned anywhere else before, let alone in Apple's documentation. As I dug around a little more, the floodgates opened and more stuff came in, including -- what a win -- the critical piece that now enables the G5 nanojit to fly. In fact, not only does it fly, it also no longer needs the shortcuts to dump those slow low-level JavaScript operations (JSOPs) because those JSOPs are no longer slow.

These are useful things to know not only for those of us trying to wring more performance out of our old Macs, but also for people using later PowerPC and POWER designs, such as Wii, Xbox 360 or PlayStation 3 hackers, because the PowerPC 970 (being a modified POWER4) is more closely related to the modern day IBM POWER systems and the Cell, Broadway and Xenon CPUs than the G3s and G4s that preceded them. So, for posterity, here's how we juiced the G5 nanojit.

The G5's data cache acts differently. The G5, because of its deep pipeline, is constantly trying to keep that pipeline full and reduce latency, and tight code trying to prime the data cache needs to be aware of the difference. The AltiVec dst (to hint the processor about where the data stream is coming from) instruction, for example, requires that pipeline to drain and can seriously impact performance. We don't manipulate the D-cache presently in the nanojit, but I mention for completeness that the more basic dcbt is preferred to dst as it does not need to be serialized; the G5 uses a 128-byte cacheline, not a 32-byte one, so dcbtl (which uses the native cacheline size) would be better still. Analogously, for zeroing out a cacheline use dcbzl, not dcbz, which only operates on 32 bytes even on the G5 and is therefore inefficient by wasting valuable cache space.

dcba and dcbi should never be used; they are illegal on the G5. Mac OS X emulates dcba on the G5 by simply ignoring it, since it's just a hint, but this causes a software interrupt to do so -- more about that in our final point.

Remember dispatch groups. The nanojit we use has a construct called the "swaptimizer," which swaps independent instructions around in such a way that more of the CPU's execution units can be running at the same time (i.e., improve instruction-level parallelism) by hoisting up instructions to run overlapped with other instructions that don't depend on that earlier step's result. In certain cases, this can be effective enough to get some instructions seemingly "for free," particularly comparisons that can write out independent comparison results (the PPC has a series of "mini-registers" for this which is quite convenient). For in-order CPUs like the G3 (remember that the G3 is essentially an evolved 603 with all the advantages and disadvantages), this is very valuable, as it only retires instructions in program order despite being superscalar, and it should also be useful for other in-order POWER chips like Xenon, the Cell PPE and POWER6. It is less valuable on the 604 and G4, which both have some limited out-of-order execution, but the G4s in most Macs don't have the reordering logic of later G4 designs.

On the other hand, the G5 is an aggressively out-of-order architecture to improve its instruction-level parallelism in hardware, and can have over 200 instructions in-flight (compared to around 30 for the G4). To reduce the amount of silicon needed to track each and every one of these flying instructions, IBM designed the G5 to take dispatch groups of instructions instead and these groups are what the CPU tracks and what travel through the pipeline. Prior to grabbing the instructions, the G5 will attempt to reorder them for maximum performance. As a result, the swaptimizer is less effective here because it's cherry-picking the low-hanging optimization fruit that the G5 already schedules for, but it does help to pack groups better so that an earlier group is less likely to need the result of a later group.

Dispatch groups actually contain operations rather than individual instructions. In most cases this is an academic point, as the operation is usually the same as the instruction. Ordinarily dispatch groups contain five operation slots: four for individual operations, and an optional branch instruction, so most dispatch groups fall between every fourth instruction. The branch instruction always terminates a dispatch group, and a dispatch group may never have more than one branch. The distinction between operations and instructions will be covered in my next point. The pieces of the group then enter the processor and are issued and executed, and the groups, not the already reordered instructions, are then retired in-order.

Because the instructions execute for most intents and purposes as a unit, certain interdependencies can really hurt, most notoriously loads and stores to nearby addresses in the same group. For example, in the LIR operation d2i that converts a double to an integer, the stfd instruction should be in an earlier group than the lwz that follows it (remember that instructions in the nanojit are emitted working down from the top of memory) because they work on memory addresses that are very close to each other. Since this could lead to a problem with aliasing if they run together in the same group, the G5 has to "undo the group" and split them apart, leading to a pipeline spill as this won't be detected until the group is formed and its addresses are calculated. It improves performance to insert a couple of nop instructions between them, which essentially act as empty space in the group, and forces the G5 to split the group ahead of time. This is tunable; our code worked best with two, and gained a small but meaningful number of points on V8. In fact, this is particularly a problem for any code that has to interconvert between floating point and integer registers, because such conversion must store in memory as an intermediate step (there are no direct FPR to GPR moves in any Power Mac processor). Shark checks for this specific situation and Apple calls it a "LSU [Load Store Unit] Reject" in ADC.

Certain instructions do better at certain positions in the group, too. For example,
mtctr (move to counter register) should be first in a dispatch group if at all possible, and I'm sure there are others (please mention them in the comments; this was the one I found in most references). We can't really leverage this because we only use CTR as an indirect branching register in constructs like mtctr bctr, so we're always branching soon after anyway. However, if you use CTR as, you know, a counter maybe 8), then you might want to group your instructions to force it first in line with a group.

Avoid microcoded and cracked instructions. I mentioned that instructions and operations can mostly, but not always, be treated synonymously. The case where they are not is if the instruction in question is cracked or microcoded.

If the instruction is cracked, it is actually two operations in one, and takes up two operations in the dispatch group; if the instruction is microcoded, it takes up all the operation slots (i.e., it can only travel alone in the dispatch group). Here is Apple's list of G5 cracked and microcoded instructions. Although these instructions do exist and do execute in hardware, they are obviously slower than other instructions. Use them only if you have to, which brings us to our final and most important point.

Never, ever, ever use mcrxr on the G5. And this might well apply to some other POWER CPUs, by the way, even though it appears in IBM's documentation. Apple doesn't mention anywhere in their documentation I could find that mcrxr is software-emulated on the G5. Yikes! In fact, simply eliminating the use of this instruction was what restored the vast majority of our speed.

The reason this hit us so badly is an oddiment of the code the nanojit generates. Some background: overflows and other math exceptions are, if the instruction requests it, annotated in a special purpose register ("SPR") called the XER. The XER, amongst other things, tracks both overflow "OV" of the last instruction, and a summary overflow "SO" which is sticky (i.e., any instruction that sets it, it stays set, unlike OV where the next instruction where overflow tracking is requested clears it). The result of XER cannot itself be used in branching: only the condition registers "CR" can be used to conditionally branch, so the CPU mirrors SO to one of the condition registers if the instruction requests it also (CR0, in case you're interested, for integer math).

The nanojit spits out lots of guard code for every arithmetic operation -- and I do mean every operation, from simple increments in a loop to computing the national debt -- so that an overflow state can be correctly trapped as an exception and appropriately handled. Most of the time, overflow does not occur, and only one instruction is being tested for overflow, so we want to use OV (or we waste cycles clearing SO constantly for instructions that are unlikely to set it). However, we can't branch on OV directly, so here comes mcrxr. mcrxr stands for move exception register to condition register, which puts the relevant bits into the CR we specify and clears those bits from the XER. In our case, this puts OV in the greater-than field of the condition register, and now we can branch on that. Problem solved! ... at least on the G3 and G4, where mcrxr is in hardware.

On the G5, mcrxr is trapped and emulated by the operating system. This means every time the instruction is encountered, there is a fault, the pipelines probably have to empty, the OS examines the instruction, sees what it is, runs equivalent code (to be discussed momentarily), and returns to regular execution. If you look at the code the nanojit generates for a simple loop, you will see that each and every time the loop runs, the increment is tested for overflow, and the mcrxr instruction is trapped and emulated. No wonder it sucked so badly! Apple doesn't mention this anywhere except if you dig through their code or find the few obscure bug reports about why a previously snappy executable on the G4 performs so badly on a G5.

So here is the one case of where a microcoded instruction is better than the alternative. We now use this equivalent code in the G5 nanojit:

mfxer r0                ; put XER into register 0, which is scratch
mtcrf 128, r0           ; put the relevant bits into condition register 0
rlwinm r0, r0, 0, 0, 28 ; clear out the relevant bits in register 0
mtxer r0                ; and put them back into XER, clearing it

You will notice that except for rlwinm every one of these instructions is microcoded and requires an entire dispatch group all to itself. There's no way around it, but it's better than triggering an illegal instruction fault and forcing software emulation. Way better. We still use mcrxr on the G3 and G4, which have it in hardware, but the G5 now uses this equivalent which is much faster on the 970. But phooey on both Apple and IBM: IBM for taking the instruction out, and Apple for not documenting it.

Well, that's enough nerdosity. It'll be in 4.0.1. It's great. More G5 docs, until Apple decides to 404 them:

Ruminations on Mozilla rapid release, the end of embedding and Firefox 4.0.1

2011-04-03T15:30:00.000-07:00

First, old business. Firefox 4.0.1 (this appears to indeed be the version number) is slowly shaping up. Assuming Mozilla is in any way typical, it will probably emerge in late April or early May. There are so far some minor stability updates, nothing especially major. There may or may not be some TenFourFox-specific fixes in there, depending on when I get them complete, but there haven't been many major issues specific to us (good!). The AltiVec WebM accelerator will not make this release; it will be in a separate beta (probably 4.0.2pre). Mozilla has stated they will continue with their "old" maintenance schedule for Mozilla 2.0, which means there will still be stability and security updates for a period of time, which is also good because of what I'll talk about next.

Before we move on, however, there is some somewhat unhappy news that Camino will drop Gecko after they release 2.1 (equivalent to Gecko 1.9.2, i.e., Firefox 3.6). This is sad to hear -- I myself was a Camino user until I started working on TenFourFox, mostly because I like and trust Gecko more than WebKit, but Fx 2.x and 3.x didn't feel like first-class Mac applications by comparison. I'd probably still use Camino if they were planning to support PowerPC, but they too will undoubtedly drop it after they end support for Camino 2.1, which in fairness to them will probably not be for many, many months.

The reason is because Mozilla is dropping embedding, at least for in-process. There is a promise in the mozilla.dev.embedding thread that out-of-process embedding will be reconsidered, but I doubt it, and for a big reason: Gecko really, really sucks to embed. It was always hard for Camino and many bugs were filed on getting it to work right (it still doesn't always), and WebKit's entire existence is owed to the fact Apple also thought Gecko would be a PITA to embed, and instead took KHTML and messed up around with that instead. Part of this is Gecko's scope, because Gecko requires XPCOM, and XPCOM isn't just some internal engine -- it's an entire object framework. But part of it is simply because Mozilla never made it a priority, and now the people who might have kept the embedding aspect alive will probably give up and work on something else. When the time comes for out-of-process embedding, there won't be any browser project around that's interested in it, and Mozilla themselves will have moved on to something else and it will never happen. This is essentially the end of Mozilla as an embedded rendering engine.

For the record, we are not embedded, neither in Classilla or TenFourFox -- we are the browser, not a shell around it, and we are built ultimately on XULRunner (or in Classilla's case, XPFE Apprunner), so this doesn't affect us. It also doesn't affect Songbird, SeaMonkey or anything else that uses a XUL-based front end, but this really sucks for people who are trying to break through XUL's interface limitations. Camino probably will survive the jump when "3.0" emerges with WebKit, but it's a real shame (especially because w/r/t custom WebKits on Mac, OmniWeb is really my personal choice and it will be hard for Camino to compete against Safari and Chrome as well), and it shows that Mozilla's priorities are nowhere near as aspirational as they used to be. Gecko was what distinguished Camino, but no longer. This is part of why WebKit will eventually eat the world, and we will damn it in the same tones we damn Internet Explorer. But I digress.

So, new business. Mozilla, freed from useful things like embedding, is busily working on Firefox 5, which right now they call Firefox 4.2 alpha. The difference in version numbers isn't too odd, as Fx4 was Fx3.7 originally. However, the major difference is more one of process than content: this will be the first release in which Mozilla will use their new "rapid release" framework. And frankly, I have no firm idea what that means, and from idly perusing the Mozilla newsgroups, there are certainly disagreements about this even among Mozilla higher-ups.

Let's review, then, what Mozilla has placed on their wiki and written by one of their key developers. The release schedule is pretty well known: raw work lands on mozilla-central, then stuff that is finished makes the cut to (these are provisional names) fx-experimental, then stuff that is shippable makes the cut to fx-beta, and then the final browser pops out into a release branch. Each stage is intended to last about six weeks, with approximately one week of overlap betwixt. Firefox 5, to jump start the process (and presumably because there is a lot of pent-up work that didn't make Fx4), will have only three weeks in mozilla-central before moving up the release ladder. That, in the words of Larry King, is what we "know-know."

What we don't know-know are several important considerations. The big one will be support intervals. Chemspills will be supported (and necessarily), but it is highly likely that releases will become unsupported much more rapidly. This makes it a lot harder for us to have a stable footing, and may indicate that security aspects discovered "late" in a release's short lifespan may go unrepaired directly by Mozilla if sufficiently late in the next release's cycle (we may have to backport significantly more fixes and forgo others when we lose source parity).

Similarly, we don't really have a handle yet on how updates will be delivered. Google Chrome has a background self-update mechanism, which has its plaudits and its pitfalls, respectively, that it keeps their audience current but also can more widely distribute serious bugs that escape into a release. Mozilla, continuing their Chrome crush, wants to do the same. From a developer perspective, this requires significant back-end infrastructure to distribute partial fixes -- we don't have this kind of back-end even for one architecture, let alone the four builds we release, including for just keeping users in sync, and would greatly complicate testing. Other Mozilla distributors probably have a similar problem. The solution is to simply build snapshots like we do now as the same sort of "big package," but this may require us to maintain a completely different release schedule. Version numbering is related to this problem. Sayre's document alludes at a way to turn automatic beta updates off if this became a reality, and we would probably ship with it hard-wired that way (just notifications, same as now).

In the meantime, I've been watching what's landing in mozilla-central for "Firefox 5" and the major thing concerning us is that IPC is now required. At one stage I did have Chromium IPC building on Tiger, but when built in debug mode, the resulting libxul (which is also now required) was too large to link. To get around this problem (and to avoid maintaining IPC), TenFourFox 4.x is built with both libxul and IPC off, but that will no longer be possible. Fortunately, I have a solution to the linker issue, and I think I can patch enough places in the current Chromium IPC to still get it to build (I'm not sure it will work, but I'm pretty confident I can at least get it to compile). Plus, the three week period makes it unlikely stuff will be "frivolously" removed that we depend on, let alone 10.5 compatibility. Therefore, the chance is not excellent, but it is reasonably good, that we will make the jump to Firefox 5 and still maintain source parity at least through Mozilla 2.2.

Assuming that we do, however, "Firefox 6" will be a major concern. 10.7 will have emerged by then, and it is quite possible that legacy code may be purged during the process of getting Fx6 into Lion. There is even an outside chance that 10.5 might be dropped and that would almost certainly doom source parity as there would be no good reason not to adopt 10.6-specific optimizations like GCD and the like. It's still too early to forecast this, but even if the worst happens early and we drop source parity later this year, like Classilla I still plan to backport security and stability features and make whatever improvements I am able to do so. After all, I still need a browser for this quad.

What that mysterious fix was: TenFourFox "4.0s"

2011-03-24T09:36:00.000-07:00

Those reading earlier entries will notice I was puzzling over two strange commits attached to a (then) sec-locked bug that forced an RC2 and appeared limited in scope. They were indeed limited in scope, but to deal with a problem that has now become public; namely, a certificate authority had credentials stolen enabling an Iranian-based attacker to sign multiple bogus certs. At least one, and possibly more, escaped.

This is one of those cases where I question why something where the exploit was, technically, already in the wild was not publicly disclosed and I would have probably held the release for it at the time.

It is now unclear exactly how much of an impact this bug has because everyone knows about the bogus certs that got released and they are already revoked at the OCSP level. Still, there is the possibility they could be used for nefarious purposes under certain circumstances and I will be releasing a "4.0s" later today with display-version 4.0 but updated build-IDs incorporating this fix. This will coincide with 3.6.16 nicely in any case. For Classilla users who follow this blog, this will be part of Classilla 9.2.2 as well.

Since I'm forcing a re-release, this will also include the more targeted fix for issue 37 which I am pretty sure is secure and stable. That should make all you JavaScript performance gearheads happy again. It will not include the AltiVec WebM accelerator; that bug is snowballing a bit and I want it to have beta coverage. You should expect these builds sometime this evening Pacific time and the G5 is already cranking them out as you read this. (UPDATE 9:10pm: Updates are now available. There will not be a separate changeset pack as, with the exception of the one-liner for issue 37, it is exactly the same and the security issue is in the Mozilla branch.)

Release postmortem and goodies in the next minor release

2011-03-23T06:31:00.000-07:00

So while we didn't have anything on Firefox 4's numbers, we had more downloads in 30 hours than we did in a month on any of the individual betas. Not bad!

The change that I landed last-minute due to the crash that cropped up does have a performance impact, but some users are seeing a hit significantly worse than my measurements. I'll talk about that in a minute. If you are one of those users, and you weren't crashing in b12, you could revert to that temporarily pending followup. Note that you will get pestered to update unless you turn that off too.

I should also use this spot to state, since it was a big concern to people in the previous entry, that plugins will have glitches. The code holding it together is kludgy at best and will eventually not work, because these plugins are no longer updated for PPC, and I don't know what the changes occurring in "Firefox 5" will do either. Right now most of the glitches are limited to scrolling artifacts. I have always suggested installing an add-on such as Flashblock, which is what I do myself, and then you can enable only the Flash applets you actually need to see. Remember, plugins will be pref'ed off by default in the next major release due to imminent security and compatibility issues.

Which brings us to the next minor release. While Mozilla is hemming and hawing about how they will set up their new rapid release schedule, and I will of course have commentary on that when they're done, we're going to continue our development on this branch. The first and most important thing will be to find a faster fix for the crashed-out JavaScript issue 37 that does not hit performance as badly (preferably at all), but the second will be to get AltiVec acceleration into WebM playback. WebM actually, believe it or not, already has this code, but Mozilla is not using it. What we'll be doing is transplanting some more of the WebM accelerated source into the tree, then modifying the Mozilla build system to properly integrate the VMX accelerator code. Note that this will not require AltiVec: G3 users can still play WebM video, just with the non-AltiVec generic C decoder that is already in use. However, G4 and G5 owners should see a boost in performance; how much I don't exactly know yet, and we might need to translate some more of it manually. We'll start with that however.

Because both of these changes are a little risky, there will be a single minor beta (I think there's a Jonathan Coulton song by that title :P ) incorporating both new updates. I hope to have that out to play with in a couple of weeks. You will not be automatically updated to it -- watch this space for more.

The good news is that other than the known issues and the JavaScript performance question, there have been no major problems with the rollout. So job well done. More work ahead.

Chemspill on 4.0 final

2011-03-22T01:06:00.000-07:00

Due to a late-breaking issue with the JavaScript accelerator, revised versions of 4.0 are now available. Note the version is the same, so if you downloaded TenFourFox earlier today or yesterday, you may need to grab it again. I apologize for the inconvenience. This version is about 10-15% slower than before, but is much more stable, and of course is still by far the fastest JavaScript on PowerPC.

TenFourfox 4.0 final released!

2011-03-20T15:30:00.000-07:00

Since we don't get the official Firefox 4, I figured, why wait for the official release date? So go get it. This release fixes a subtle but nasty crash glitch in the JavaScript accelerator, fixes some more Command-shortcuts once and for all, and should also resolve trouble with window stacking. I've been using it myself on my G5 and iBook G4 for the last week and it's great. You'll love it too.

I should add one note about the build: before I left for my mini-vacation, Mozilla had been planning to make RC1 into 4.0 final, and I left the G5 buildhost to chug out the various architectures while I gallivanted around the coast. Mozilla then made a quick late-breaking announcement and popped out an RC2 with these fixes for bug 642395. Don't bother trying to look at the bug as it's sec-locked and I'm not part of the Mozilla security group either; just look at the diffs. It appears these changes are to temporarily blacklist certain certificates issued by Usertrust and I don't know why yet despite some cursory snooping around. The nature of the fix suggests this is temporary and of limited import, so rather than invalidating a whole lot of build work which has already been certified, I have elected to push out TenFourFox final relative to RC1 (those marked changes are the only difference between RC2 and RC1) unless I discover that the issue in question is more severe. If these patches are still in the tree by the time the next maintenance security/stability release comes due, then they will become part of TenFourFox also.

Which brings us to our next point: will there be point releases for Firefox 4? Signs point to yes. This is very good news, because I am hoping that Cairo 1.10 lands on mozilla-2.0 and helps our software rendering performance (see issue 7). If nothing else, it keeps us stabilized until we must do battle with Firefox 5. That will be the subject of a post to be made in the very near future.

For builders who want to build on this new stable (ho ho ho) basis, there are now new build instructions and the changesets for 4.0 final are relative to mozilla-2.0, not mozilla-central (although necessarily they would probably still largely apply there too). You probably want to clone mozilla-2.0 separately if you want to make your own builds off that stable branch and use those changesets. New changesets for "Firefox 5" are a totally separate topic, but assuming that we make the jump to Fx5 (again, a subject for a post coming soon), I will not be popping out separate beta changesets until Fx5 has itself entered beta -- i.e., no alpha changesets, sorry, unless there is significant demand.

In security news, Adobe Flash is once again being actively attacked. Adobe's advisory implies, but does not state clearly, that a fix will be issued for 10.1 (or PowerPC). This fix is supposed to come out sometime this week. If Adobe does not fix this for PPC, then watch out. Remember, plugins are deprecated in 4.0, and will be pref'ed off in the next major release. Get your Flashblock on now; it works fine with TenFourFox.

Anyway, go grab TenFourFox 4.0 final and make sure to flaunt it in the faces of your weenie Intel Mac-using friends. They'll be oh so jealous. ;)

NecesRC delays and living la vida stable

2011-03-07T21:38:00.000-08:00

Mozilla has spun mozilla-2.0 off into its own stable branch to generate the Release Candidates. Congratulations, we made it! Whatever else happens, we are almost certain to continue surviving through future versions of Firefox 4/Mozilla 2, as it is highly unlikely that Mozilla will throw one of our known show-stopper insurmountable technical hurdles into a maintenance branch. (There is an early development hurdle with Firefox 5 which is going to present a problem, but I'll discuss that in due time. Fortunately, I am reasonably confident that this technical requirement can be worked around.)

Now that we are on a stable branch for sure, updates will be handled a little bit differently. Previously I would jump on the tree as soon as it was held for tagging by Release Engineering so that no or very few bugs would land that should not be in the beta release. Since our QA demands are much less than Mozilla's (only two supported OSes and one supported architecture), building, testing and certifying releases usually only took a day or two after that, and so routinely we were popping out betas sometimes as much as a week or more before the official betas would come out. Well, no longer. As I alluded to in the last post, it behooves us to wait as long as possible on a stable branch to make sure everything's on the branch that needs to be there, because the risk of waiting too long and getting an unstable patch in the pull is lower than the risk of jumping on the tree too early and missing a security or stability issue that lands at the last minute.

For that reason, 4.0 final and all subsequent releases off the stable Mozilla 2.0 branch will be released pretty much simultaneously with Mozilla's releases so I can make sure that changesets match. Unfortunately, you can't do this with about:buildconfig because the changeset displayed is based on the internal TenFourFox repo and not Mozilla's, but we can do this internally with a bit of Mercurial magic on the build system. Therefore, don't expect 4.0 final much before the official release day, nor any future 4.x release. We'll still be punctual, just not fashionably early. :)

Builders and our Fink-based audience will be happy to know that when these stable releases go out, they will also be accompanied by new changesets against mozilla-2.0 instead of mozilla-central, as I know the current changesets do not apply completely cleanly to the trunk repo. On a stable branch they should have much fewer variances and therefore ought to apply without much fuzzing. Our internal copy of mozilla-central will also be rebased on this clean changeset too so that everything is consistent. I plan to do this for each new branch in the future so that there will be much less junk metadata getting carried forward.

There are a couple of 10.4Fx-specific bugs that should also be fixed in 4.0 final, none serious showstoppers, but might as well get them in before the end as the changes should be very safe. In fact, I have preliminary fixes in my tree already and have been using the "RC" on my G5 for the past few hours now. Looks good so far; later this week I'll dump a build on the iBook G4 and see how it does with that.

So, champagne and drinks all around: we made it! Firefox 4 is still on the Power Mac and still on Tiger! I have some ideas for more PowerPC-specific optimizations for TenFourFox 5, too, but more on that once we branch.

Beta, shmeta, real men release

2011-02-24T21:13:00.000-08:00

So here's what's going to happen next.

There will be no beta 13. Even if Mozilla themselves do one, which seems unlikely but one never knows, we will not. There isn't enough there to justify it unless someone can demonstrate a serious blocking bug in beta 12.
We will not bother with the Release Candidates. These are, at longest lived, highly ephemeral releases and every release we put out is another 70-80MB out of our Google Code hosting space. Releases this temporary are not a good use of our currently limited community resources.
Once blokking boogs go to zarro, we will watch to see what Mozilla does next: if they immediately branch into a new mozilla-2.0 repo, then we will build off the repo after the branching. If they do not branch then (though they will branch eventually), we will continue the "advance" pulls we are doing now to avoid getting post-tag commit noise until they do.
Once Firefox 4/10.4Fx goes into its own separate repo apart from mozilla-central, pulls and builds will only occur after everything has landed (as these maintenance repos are historically much more stable than the trunk, as you'd expect), which means our releases will be either on time or slightly after the fact. After all, Mozilla never tells me anything. We'll do our best to be timely, however.

No ETA for any of this yet. The number of bugs to get to RC is not massive but some issues are non-trivial (though getting smaller all the time), and my best estimate gives formal release no earlier than the end of March.

Beta 12 released!

2011-02-23T17:57:00.000-08:00

Before you all write and tell me, yes, I know the build says b13pre. That's because there was a last-minute security bug that landed and by the time I jumped on it, the build milestone had already been advanced. There's no real purpose in a checkin-and-out to simply massage the version number in a beta, so I left it as was. However, despite the version number, this is beta 12.

Okay, enough of that. Beta 12 has several new features and these are mostly good, some strongly so. First of all, the most visible change is that the hovered URL in the location bar is gone, as threatened, replaced by a Chrome-like strip at the bottom (the same one used to report network activity). I kind of miss the location bar layout, but I adapted back to the old style very quickly. The bar will move to the other side if you hover over it, so it doesn't get in the way. Second, there are small subtle changes (again) in tabs, including the close button, which is now just a tiny grey X in the default theme. The bogus new change is that the content-specific modal dialogue boxes are now super blah (I liked the quasi-Aqua ones in beta 11 better -- these are boring).

In the systems department, JavaScript is also improved -- my iBook/1.33 clocked a 400ms improvement on SunSpider, and Tab Candy is a bit sprightlier -- but the big one is memory pressure. Mozilla has been tracking some leaks in Firefox 4, some minor, some major, all bad, and a big improvement landed in the form of bug 630932. This bug dramatically increases the aggressiveness of the garbage collector and cycle collector, and for those of you like me with memory monitors in the menu bar, you can see the GC/CC quickly grabbing RAM back as you close windows and tabs. TenFourFox has more memory pressure than Firefox did, not just because of Gecko 2, but because we generate and cache machine code for the nanojit. Anything that helps to actively reduce footprint is welcome and this makes a big difference. For those paranoid about memory, here are some other memory leaks Mozilla is tracking, and some of these have fixes planned for final release.

Mozilla also made some significant changes to the plugin architecture on Mac, as I warned you about. I was able to hack around them (I think), but the time is coming when I might not able to. Flash performance seems slightly worse because of the changes (which favour CoreAnimation and hardware acceleration), but it's hard to get accurate metrics. I still advise downloading such videos to disk with any of the available video site/YouTube extensions and playing them outside of the browser.

Specific to TenFourFox, we have a few fixes landed in this release for a new problem and for a couple old ones. One of our oldest and most frustrating bugs was the awesome bar refusing to suggest anymore when you typed a totally novel URL it could not suggest for: when the box closed, it would not open again (though it could still receive keyboard events). I'm inclined to chalk this up to 10.4 being a brat who can't stand not knowing the answer, because I could not find a specific cause, and tweaks to the underlying object had side effects for other auto-complete objects. So, if you can't fix the cause, remove the situation. Starting now, if you type a new URL that the awesome bar doesn't know, rather than simply closing, it just shows a single blank box (actually a single blank stub suggestion). This works with the interface, and the awesome bar works normally. If someone else has a better idea, say so, or send me source code. :P

Also, for those of you who noticed some Command-key shortcuts weren't working, that should now be fixed, and a crash in the JavaScript nanojit was also wallpapered over. So what are you waiting for? Go get it!

I'll post later this week or early next what will happen for the final formal release. Until then, enjoy beta 12.

Coke is it. I mean, beta 12

2011-02-11T22:56:00.000-08:00

Beta 12 is going to be the last Firefox 4 beta, so it will be the last TenFourFox beta too. There will be a couple extra 10.4Fx specific fixes, hopefully.

It's unclear when Fx4/Gecko 2 will get its own hg branch. Once it does, this will start the countdown to see if we jump to Firefox 5 with them on mozilla-central. Remember, it's not guaranteed that we will retain compatibility, and there is always the risk they could require a 10.5-only feature at any time, though a serious widget library incompatibility isn't likely to happen until they require 10.6.

Once Fx4 moves to a stable branch, changesets will be issued against that stable branch, and the unstable branch will be idle for an indeterminate period until Firefox 5 goes through a sufficient number of betas. More about that when it actually happens, and knowing Mozilla, I don't see this happening much before summertime.

By the way, Flash 10.2 came out, and as threatened it is Intel-only. It is unclear if there will be any more 10.1 fixes, and to even get 10.1 you have to find their archived Flash Player versions page. I'd advise you don't hold your breath over continued support. Flash Player 9 has been completely deprecated, so if you must use Flash, use Flash 10.1.

Beta 11 released!

2011-02-02T20:43:00.000-08:00

Go get it!

As promised, the key grab is JavaScript acceleration now enabled for all platforms. If you get weirdness, check it by turning javascript.options.tracejit.content to false (and preferably restarting your browser). If that fixes it, report the site. Note that because the nanojit stores traces in memory for future cache hits, memory usage in this beta will be higher -- expect it. If this is a serious issue for your memory impaired system, turn that pref off as well. I also threw in a little helper for Gopher addicts like me, did some work on the title bar, and tried to work on the widget code a bit to help windows sort better.

Mozilla has, IMHO unadvisedly, decided to do some late-breaking significant interface changes. These changes are only partially in beta 11; the most noticeable is the status notation at the lower left during network exchanges a la Chrome. This is fine, no worries, and even welcome. However, Mozilla is planning to redo the location bar and remove the hovered URL, restoring it to its former location or something approximating that, and all that just as I'd gotten used to it in its current place. I suspect this will confuse users all over again who, like me, had become accustomed to the current layout; adding insult to injury, for whatever reason this does not seem to have made the cutoff for b11, and that's not a good thing either because the inevitable screaming about this needs to be dealt with early and not reserved for beta 12 (or 13). Nevertheless, expect some alteration in the next beta.

Mozilla is getting better about their time-based delivery and their estimates are becoming more accurate as the formal release nears shipping (I estimate mid-March). The big question is if there will be a beta 13. If there will not be, then we will release beta 12 and go formal. If there will be a beta 13, we will skip 12, release 13, and go formal. Heaven help us if there's a 14.

For builders, changesets will be up later tonight or tomorrow morning.

Beta 11 builds in progress

2011-02-01T21:43:00.000-08:00

Beta 11 is now in the pipeline, now that Mozilla has signed off on the green rev, and I'm running my quad's fans at full blast churning out builds tonight and tomorrow for release later this week.

I'll speak more about the release in a bit, but the big ones are additional performance improvements (both in display speed and baseline JavaScript), and also that the nanojit will be enabled for all CPUs, including G5. The G5 nanojit does not gain performance wins as massive as the G3 and G4, and it's still net slower in SunSpider. However, it is net faster in Dromaeo and V8, and the browser just feels faster with it. This was done initially by blacklisting JSOPs that empirically seemed to hurt, while keeping as many of the winner JSOPs as possible, and the result is best expressed in terms of Meat Loaf ("two out of three ain't bad"). I think I can do better here by getting even more low-level, but this is a solid start and gets all of the supported processor families at feature parity.

Because this version has G5-specific code paths, people working on their own builds should note the change in the recommended configuration file. In particular, you may need to add --enable-tenfourfox-g5 to it and/or define TENFOURFOX_G5 in files you want the alternate code path to be used. While this is only in the tracejit currently, and only then for G5/970, I may further expand this for more processor-specific code-level optimizations in the future. I'm also looking at adding processor macros independent of TenFourFox's specific settings, which will most likely be done at the compiler level (for example, the G5 version now also sets -D_PPC970_ for gcc). This will help others who want to use this code for non-Mac PowerPC systems.

More on the specific build in a couple days once the binaries are certified, but as a heads-up, it appears there will be a Firefox 4 beta 12. I am undecided if we will push out a b12 also or hang tight for the formal release, but there are still a lot of hardblockers and as usual I suspect there might well be a beta 13. How unfortunate, because as you can see,

... Firefox just wants to be released! Sorry, that was too cute not to share. :)

(image courtesy @philikon)

Their betas go to 11

2011-01-21T06:40:00.000-08:00

Mozilla wants to freeze beta 10 this Friday and push it out next week, then freeze beta 11 the Friday of next week, and push beta 11 out the week after that (see this announcement). Besides my moment to say I told you so, two back-to-back betas in as many weeks is not good use of our more limited resources. If Mozilla pushes out beta 10 on schedule, which may in fact happen based on some of the rumblings I hear around Mountain View, then we will sit out beta 10 and go to beta 11. If beta 10 is delayed longer than next week, we will continue with beta 10. Watch this space or mozilla.dev.planning.

Plugins unplugged

2011-01-19T13:23:00.000-08:00

For those of you who use Classilla on OS 9, you already know that Classilla allows you to turn plugins off, and in fact encourages it. It makes the browser more secure by not exposing old unmaintained native code, and it makes the browser more stable by not exposing old unmaintained broken code. If you really really really want to use Flash Player 7 to view something, or QT 6.0.3, you can, but you can turn it right back off again. When I am in OS 9, I just keep plugins off entirely. Plugin support for OS 9 has obviously not been updated in years.

The time is now coming for this state on PowerPC. Adobe has been making noises about ending PPC support for the Flash Player plugin for some time now, and their preliminary system requirements page for Flash Player 10.2, which is now in release candidate stage, finally makes this plain: PowerPC is no longer a supported platform for Flash, although they still support Tiger on Intel. When you download Flash 10.2, it's x86 or bust, similar to their dropping PowerPC from AIR support with 2.0.

Most of us know that Flash was never well-optimized for PowerPC anyway. Even on a fast G5 it chugs, and you can forget about full-screen or HD playback on G4s, let alone G3 systems. Losing Flash is not going to be a massive issue for functionality for awhile because much of the advanced functionality 10.2 introduces will not be widely adopted until 10.2 is. However, it is going to be an issue for security. Flash is a large potential and practical security hole, and while many of the exploits fortunately rely on x86 native code, some do not. Moreover, Flash-based privacy intrusions, while not bugs or security holes per se, have for a long time approached infamy in quantity and ubiquity. While my insiders tell me that Flash 10.2 is light-years ahead of 10.1 in performance, that certainly does not help us on our minority soon-to-be-unsupported architecture.

While Flashblock (at least until beta 9 ... fix it!) helped to block most Flash applets, and tools like AdBlock and NoScript can also serve in this capacity, the time is coming where TenFourFox will go the way of Classilla and block Flash and other plugins at the layout level. This has obvious benefits for security and performance, at the cost of losing Flash. Steve Jobs, bless him, was right to say that HTML5 supersedes many of the features Flash has long been (ab)used for -- not all, but many -- and I think that if we deal with this pain now, we will be better able to adapt. More to the point, Mozilla is moving towards plugin and graphics NPAPI models that no longer use QuickDraw (deprecated since 10.4, and almost certainly further marginalized in 10.7), and this will be an issue for 10.4 which lacks CoreAnimation. In the future it may not be technically possible to still use Mozilla's code base and maintain compatibility with the aging plugin base on PowerPC and Tiger, let alone the attendant disadvantages of continuing to use those plugins.

This post is more of a warning rather than an announcement. While I plan to ship TenFourFox 4.0 with plugin support, albeit deprecated, likely by 10.4Fx "4.1" plugins will be pref-ed off requiring you to manually re-enable them through about:config. At some point in the future when it is no longer technically feasible to support them, that code will be completely removed. Hopefully by then the impact of iOS will have introduced a world where Flash-free alternatives that are natively supported by TenFourFox and Gecko 2.0 are easily accessible.

Lies, damn lies, statistics, and the G5: a beta 9 postmortem

2011-01-17T21:49:00.000-08:00

Beta 9 has been an ... interesting release. Besides a serious blocker that escaped Mozilla's notice causing large portions of some pages to not render at all (bug 623852), and a flickery title bar which is also a Mozilla bug (partially fixed for Tiger in issue 16; see also bug 621762), we are also seeing significantly faster screen display times and the first community statistics for the nanojit.

For those who have been sleeping under a bridge and not reading this blog or the last couple entries, the nanojit is a component of Mozilla's JavaScript interpreter that turns execution "traces" into machine language, a limited and specific form of Just-In-Time compilation (hence the name). When the trace is executed again, the compiled code is run instead, which -- at least in theory -- should be significantly faster. The last couple entries go into this in nauseating detail, so read them if you'd like to get up to speed on the concept.

The nanojit was developed on my quad G5, and the timings were done on it in the expectation that the G5's clock speed, out-of-order execution hardware (being a POWER4 disguised as a PowerPC) and fat memory bandwidth should enable it to be the fastest performer. Full-tilt, running at Highest power, the native, non-accelerated JavaScript interpreter in TenFourFox b8 and b9 chugs through SunSpider in around 3500ms. This is about 200ms faster than Firefox 3.6, and is clearly the fastest pure-interpreter score of any Power Mac in Firefox. When the nanojit is turned on for the G5, however, this number actually gets worse -- to around 5000ms. On Dromaeo, the nanojit is slightly slower overall, dragged down by the SunSpider score. Given that the G5 has all this awesome hardware in-core to execute code as fast as possible, I concluded that the nanojit would not be as valuable on PowerPC as it has been to other platforms as the G3 and G4 don't have that extra silicon. I left the code in there for people to play with and turn the pref on or off (see the Release Notes for how to do so), since Dromaeo showed it to be valuable in certain cases and perhaps I could deal with them predictively.

Well, the tables are turning over. Thanks to user PoLiYa who wrote in to tell me about his fabulous score on SunSpider with his PowerBook G4/1.5GHz and the nanojit turned on: a whopping 2866ms, down from around 6400ms. That's not a typo; his 1.5GHz G4 benched faster than a 2.5GHz G5 on SunSpider with the nanojit. I thought this had to be a mistake, so I got out my iBook/1.33GHz and repeated the test. It was no mistake. On the G4, the nanojit not only worked, it easily cut SunSpider scores in half, and Dromaeo was also about double the speed. User agg23 over at 68KMLA tried it on his G3, and also saw a similar speed improvement, from 16000ms down to around 8900. Try it yourself and post your stats in the comments.

This does have a penalty to pay: since the trace only pays off when it's cached, the browser does indeed cache it to take advantage of it as much as possible, and this in turn causes greater memory pressure. Benchmarks really make the increased memory usage noticible, but even stock Firefox can bloat a little with regular use despite its aggressive garbage collection algorithms. This was never an issue for PowerPC Firefox before, because it never had a nanojit, but now it does. (This does not happen when the nanojit is disabled, which is the default state in b9 right now.)

There is at least one bug in the nanojit I am aware of that can cause crashes or weird behaviour, but it is infrequently triggered and I have it fixed for beta 10 already.

So, now that we know it is highly beneficial for all but the G5 processor, the plan for beta 10 is to ship the G3 and G4 versions (7400 testing is pending) with the nanojit turned on for content (web pages). If this works, it may also be enabled for chrome (i.e., the browser's own JavaScript, and add-on JavaScript), but we'll start small to make sure there are no unexpected surprises.

Where did the G5 go wrong? The most reasonable explanation is to conclude that the G5 pays a heavier penalty for stalls and/or memory access than the other processors, and the PowerPC nanojit's built-in optimizer negates the 970's out-of-order advantage because it is already putting the instructions in as optimal an execution order as possible (given the relatively poor optimizability of the code to start with; see the blog entry two back about that). Indeed, there are quite a few stalls in Shark when you load the generated code into it and analyze it, but these stalls occur for the G3 and G4 too; therefore, they simply must hurt more on the G5, and the longer pipeline is likely to blame. It is worth noting that the G5's pipeline is even longer than the POWER4 from which it descends, purely for purposes of faster clock speed. Coincidentally enough, I recently acquired a POWER6 as a household server and it will be interesting to see how well it does with this code when I get Firefox 4 compiled on it. As it stands right now, however, the nanojit will be disabled for G5 by default. In the near future I will experiment with modifications to the JavaScript tracer algorithm to see if I can predict those specific code situations where the nanojit pays off for the G5 too. You can still turn it on, but it will be shipped with the prefs off in that specific build.

Assuming Mozilla does not come out with a beta 11, beta 10 will be the final TenFourFox beta. I'm not planning to release a release candidate; we don't have the space, and I'm not all that certain b10 will be the last Firefox beta either. If it is, we will follow up b10 with a formal release and welcome the masses to our fold. Tell your friends: we're bringing our beloved old Macs into the next browser generation. More innovations to come.

Beta 9 now available

2011-01-12T21:42:00.000-08:00

Beta 9 is now available (changesets coming, hang tight). Mozilla is doing more work on the graphics and layers code, which is visible in a couple sites that for me have broken (and I will be filing reports), but also in that overall drawing performance is in general better. This is important because 10.4Fx does not support hardware acceleration, so anything that gives us a win in software is precious.

After working more with the nanojit, I have decided not to make it the default, but it is there, and you can use it. Go into about:config, under javascript.options.tracejit.*, and set either or both of chrome and/or content to true. Restart the browser. You'll find many things are faster, but some things are still slower, and there also were occasional crashes when quitting the browser: more of a nuisance than a serious problem, since you were quitting anyway, but still not very smooth. If you have good knowledge of PowerPC, I'd love your insight into improving it. The nanojit is in the changesets, but I have now submitted it to Mozilla and you can also read the diff on Bugzilla if you like. Feel free to comment.

Other things in beta 9 include a less flickery title bar (hopefully), which was a very annoying thing that cropped up in beta 8. Although it was an infrequent irritation on my G5, some users got really walloped by it. Mozilla fixed some part of this for Leopard, but that fix doesn't help Tiger, so 10.4Fx b9 has a separate rendering path for 10.4 that uses a naughty kludge. Because of this, the title bar does look ever so slightly different in 10.4 than 10.5, but only if you're looking for it. The title bar still does not always match the window gradient, but this is part Mozilla's bug and part ours. If you are really anal about that, install a Persona; they always work.

The UI was also adjusted to bring the app to the front when a window is clicked on. I am dithering over forcing the window up, or just forcing the app up, so that you can type but keep your window ordering -- I'm leaning towards the second, but will take arguments for the first. You can always click the Dock icon to bring the whole application and all windows forward, and this fixes the bug in beta 8 where sometimes it didn't bring anything forward when you clicked on a background window. It uses Carbon, don't tell Steve! Oh, and find Sluggo! We miss Power Computing! Classilla users will have seen an earlier version of this easter egg before. No fair reading the changesets to find the magic words.

Note that Flashblock as of this writing is not yet upgraded for beta 9, which should be a oneline change for them. If you really need it (and most of us do prefer it), you might want to wait until it is, or use an equivalent.

How the bench was run, and what it got us*

2011-01-06T21:59:00.000-08:00

*apologies to Michael Stipe

First some laundry list stuff. I've heard rumours about a Fink version of 10.4Fx, which sounds great. I see fangism is following this blog and I think this is his work, so ping me in the comments and maybe we can get your info into the project wiki.

Second, beta 9. Mozilla is apparently shrinking the scope for it, which is a good sign in that they are probably approaching release candidate state finally. At this rate, we should have Firefox 4 just in time for the USA presidential election. ;) This coincides nicely with the release of the nanojit, so there will be a beta 9 since it is convenient to do so. Mozilla's time frame is shooting for a code freeze, like, tomorrow, though historically their estimates have been at times wildly optimistic. That said, however, if their forecasted schedule is in any way accurate our beta 9 may lag it slightly instead of being ahead as usual. I'll explain, but first, the nanojit.

I implied in my last post that the nanojit is unfortunately not the dramatic leap I was hoping for (notice that I did not say how the bench[mark] was won), and those of you who know me at 68KMLA have probably read my post on that subject. To understand why this is a mixed bag, I should explain what the nanojit actually does, which is also useful to Mozilla hackers at large since there is precious little MDN documentation on it.

When the tracer thinks it has hot code, it turns on a trace recorder and slurps up the JavaScript opcodes as they come down the "pipeline." These opcodes are not executed as is (as the interpreter would), but are turned into an intermediate language called LIR. LIR is also the intermediate language for Flash ActionScript, as they are both part of Tamarin, though SpiderMonkey (Mozilla's JavaScript implementation) does not use other parts of the ActionScript interpreter.

After an initial spew of LIR opcodes and/or optimization, the opcodes are then handed to the nanojit assembler (in the source tree, this is in js/src/nanojit/Assembler.cpp). This section is more or less architecture-agnostic, which is important later on. It goes through each of the opcodes and as needed to generate code calls specific portions of the architecture-specific nanojit (in our case, NativePPC.cpp) which then write instructions to memory, starting with a standard function prologue. This is not done using things like __asm__(), but using a collection of macros and an opcode table to generate 32-bit instructions directly. If there is some concern about the result of an operation, a guard may be inserted as an internal consistency check, and side exits are made available to abort to the interpreter if funny business is detected. Finally, at least the first time through, an epilogue is emitted. As more code is written and more labels are fixed in memory, the assembler will go through and patch the branches in much the way a two-pass assembler would, eventually terminating in the epilogue that was originally emitted.

When it comes time to execute, the tracer makes certain statistical decisions about how useful the trace will be relative to simply falling back on the interpreter and runs one or the other. If a trace is deemed to be detrimental or pathological for performance, it is blacklisted. Otherwise, once it is compiled, it is used as long as it remains valid or until garbage collection destroys it, the rules of which are still not yet determined for Firefox 4.

As originally written, the PPC nanojit worked fine for Tamarin, but was not sufficient for Firefox. It did not have an implementation of nFragExit, which is needed to manage linkages with other code fragments and the epilogue, nor could it deal with the situation of a branch with a 14-bit or 24-bit displacement needing to be patched to a larger displacement (i.e., turning simple branch instructions into things such as mtctr b(c)ctr), and it did not implement the overflow arithmetic instructions, which use guards to catch math overflow and are heavily relied upon by Mozilla.

My first pass, then, was simply to add those features (I am grateful to Edwin Smith's help here, the original author, who gave me the critical pointers to get started). This was enough to get the browser up and running, but it didn't seem much faster, and SunSpider numbers were disastrous: on my quad G5 in Reduced power, the benchmark took around 7000ms on the stock interpreter in beta 8, but on "TraceMonkey PPC" it took nearly 10000ms -- almost half again as long! Dromaeo was less discouraging, and it did extremely well on the microbenchmarks, but its aggregate score was seriously hurt by the SunSpider dependency causing it to still be slightly slower in the final summation. (Please note that no PowerPC browser with the possible exception of Safari 5 on Leopard can run Kraken without having UI stalls, so I didn't bother.)

The suckage was, fortunately, proportional. There was no specific penalty paid other than that of the tracer and assembler themselves, and using Apple's built-in MakeDataExecutable() function -- which is still part of Carbon, but is only available on PPC -- to flush the instruction cache was actually faster than a hand-rolled routine. Less complex routines ran in proportionately less time, so the problem was the code it was generating.

It is instructive to look at the example of a simple loop:

var i, j = 0; for(i=0;i<2999999;i++) { j += 2; } print(j);

This can be written extremely fast in PowerPC assembly, especially being a single loop that can easily take advantage of the CTR branch instructions to do multiple operations simultaneously. Indeed, as a ceiling test I handwrote this into assembly and fed it to gcc, and the result was too fast to time accurately. However, when compiled by the nanojit, it took around 5000ms (!). Here's what it generates, edited for space and clarity. You can see this yourself by building a debug version of TenFourFox and starting it up with the environment TMFLAGS set to native (there are loads of other options, set it to help to see them):


/src/mozilla-central/obj-ff-dbg/dist/TenFourFoxDebug.app/Contents/MacOS/% ./js -b -d -j -p ~/empty.js
[ ... ]
0x3565ed0   [prologue]
0x3565ed0   mflr r0
0x3565ed4   stw r0,8(sp)
0x3565ed8   stw r31,4(sp)
0x3565edc   mr r31,sp
0x3565ee0   stwu sp,-144(sp)
; single target patch 03565fd4 -> 03565f30 [ I'll explain this in a moment ]
0x3565ee4   stw r13,-72(r31)  <= spill r13
0x3565ee8   stw r14,-68(r31)  <= spill r14
[ .. rest of the prologue .. ]
0x3565f28   stw r30,-4(r31)  <= spill r30
0x3565f2c   stw r3,-76(r31)  <= spill state
0x3565f30   [label1] ; the actual meat starts here
0x3565f30   lwz r28,8(r3)
0x3565f34   lwz r29,12(r3)
0x3565f34   ; pc now 03565f34

0x3565f38   ; swapped @ 03565f38

0x3565f38   lis r30,801 (0x3210000)
0x3565f3c   ori r30,r30,57368 (0xe018)
0x3565f40   lwz r30,0(r30)
0x3565f44   cmpwi cr7,r30,0 (0x0)
0x3565f48   beq+ cr7,0x3565f58
0x3565f4c   b 0x3575dec ; this goes to an exit block
0x3565f50   nop
0x3565f54   nop
0x3565f58   lwz r30,1560(r29)
0x3565f5c   stw r30,0(r28)
0x3565f5c   ; pc now 03565f5c

0x3565f60   ; swapped @ 03565f60

0x3565f60   li r27,2 (0x2)
0x3565f64   stw r27,8(r28)
0x3565f68   addo r30,r30,r27
0x3565f6c   mcrxr cr7
0x3565f70   ble+ cr7,0x3565f84
0x3565f74   lis r0,855 (0x3570000)
0x3565f78   ori r0,r0,24140 (0x5e4c)
0x3565f7c   mtctr r0
0x3565f80   bctr
0x3565f84   stw r30,1560(r29)
0x3565f88   lwz r30,1552(r29)
0x3565f88   ; pc now 03565f88

0x3565f8c   ; swapped @ 03565f8c

0x3565f8c   li r27,1 (0x1)
0x3565f90   addo r30,r30,r27
0x3565f94   mcrxr cr7
0x3565f98   ble+ cr7,0x3565fac
0x3565f9c   lis r0,855 (0x3570000)
0x3565fa0   ori r0,r0,24236 (0x5eac)
0x3565fa4   mtctr r0
0x3565fa8   bctr ; this also goes to an exit block
0x3565fac   stw r30,1552(r29)
0x3565fb0   stw r30,0(r28)
0x3565fb0   ; pc now 03565fb0

0x3565fb4   ; swapped @ 03565fb4

0x3565fb4   lis r29,45 (0x2d0000)
0x3565fb8   ori r29,r29,50879 (0xc6bf)
0x3565fbc   stw r29,8(r28)
0x3565fbc   ; pc now 03565fbc

0x3565fc0   ; swapped @ 03565fc0

0x3565fc0   cmpw cr7,r30,r29
0x3565fc4   blt+ cr7,0x3565fd4
0x3565fc8   b 0x3575f0c ; and this also goes to an exit block
0x3565fcc   nop
0x3565fd0   nop
0x3565fd4   lis r2,0 (0x0) ; this is patched by the single target patch above
0x3565fd8   ori r2,r2,0 (0x0)
0x3565fdc   mtctr r2
0x3565fe0   bctr
0x3565fe4   b 0x3575f6c
[ .. skip memory to epilogue .. ]
0x3575f6c   lwz r3,-76(r31)  <= restore state
0x3575f70   lwz r13,-72(r31)  <= restore r13
[ ... ]
0x3575fb0   lwz r29,-8(r31)  <= restore r29
0x3575fb4   lwz r30,-4(r31)  <= restore r30
0x3575fb8   lis r12,1152 (0x4800000)
0x3575fbc   ori r12,r12,31964 (0x7cdc)
0x3575fc0   lis r2,855 (0x3570000)
0x3575fc4   ori r2,r2,24528 (0x5fd0)
0x3575fc8   mtctr r2
0x3575fcc   bctr
0x3575fd0   mr r3,r12
0x3575fd4   mr sp,r31
0x3575fd8   lwz r31,4(sp)
0x3575fdc   lwz r0,8(sp)
0x3575fe0   mtlr r0
0x3575fe4   blr

Ugh!

For the sake of fairness, I have left the guards out as they have a very minimal impact on runtime, and the prologue and epilogue are fairly standard ABI stuff. But this is a lot of code for a small loop!

As originally written, the branches were not hinted, and my first attempt was to hint the branches with likely bits and allow it to optimize a patch down to a smaller set of instructions where possible. Notice the code near the end that loads CTR with 0x0 and tries to jump to it -- that is recognized by another nanojit function called nPatchBranch that (supplies!) patches branches as a long jump and the correct target is inserted into the lis-ori by the patch routine. If later in runtime the branch had to be changed, if it could be turned into a 14-bit branch with fewer instructions, it would be and the rest simply nopped out. Likewise, the nops in the code are there for room if a branch has to be "demoted" to a slower far call. This didn't help much, so I loaded the code into Shark.

Shark showed several things of note. First, as you scan the code, you will notice lots of potential pipeline stalls (ignore the swapping comments, I'll get to that). While the G5 is fairly aggressive with instruction reordering, it is not perfect, and older CPUs are less effective. Second, the branches are really hot. Third, there are a lot of store instructions inside the loop body proper aggravating the stall situation. Fourth, it's decrementing the loop the hard way.

To help with instruction ordering, I rewrote the macro that emits instructions to see if it could dynamically reorder them with the preceding instruction (call it "peephole optimization" if you like). You can see where the swapper worked in the instruction stream (just swap in your mind the instructions immediately around the swapper's log note). If there were no register dependencies and the code did not need to be emitted in strict serialized order, the instructions were swapped, which helped to turn things like


li r0,...
ori r0,...
li r1,...
ori r1,...

into a more typical and efficient


li r0,...
li r1,...
ori r0,...
ori r1,...

and cool off the registers. Although the G5 could do some of this in hardware, it helped with the G4s. A special attempt was made with comparison instructions to turn st cmp b into cmp st b so that the comparison would be more likely to be computed by the time the branch came along, and a store instruction could be safely switched with it since it would not affect the comparison. Finally, expensive instructions such as integer multiply were strength-reduced into repeated smaller operations (in mulli's case, into repeated strategic adds). These improvements yielded about 5-10% additional performance.

Still, the really ugly instructions such as branching could not be totally eliminated, and the memory stores could not be moved out of the loop because they stored stuff in the runtime stack that the side exits (if they were taken) might require. This was really painful for a processor like the PPC that is strongly oriented to load-store (whereas x86 is much better at memory operations) and had pipelining consequences. There is no way right now for the PPC nanojit to get a bird's eye view of what it's compiling so that it can more efficiently order larger functional blocks, such as moving the stack store operations into a side exit or the epilogue, or rewriting loops with the special PPC loop instructions if that sort of situation can be recognized. Perhaps that is more the domain of the methodjit, but it would be nice to have such meta-analysis available for the nanojit as well.

It could hurt less if the tracer were statistically able to predict accurately for the PowerPC which operations would be more painful for runtime, but tuning these parameters yielded small margins to date.

So am I throwing in the towel with the nanojit? No. For one thing, it still is faster at certain operations, and it can cache code it compiles (which is beneficial to L2-fat processors). While code would have to execute a large number of times to be worth it, things like the browser chrome do indeed execute a large number of times, as do many AJAXy applications. For another thing, I want to get more eyes on it to see if we as a group can make it better. While trying to beat a powerful optimizing compiler with a kludgey code generator is generally a losing proposition, there are probably other optimizations I haven't thought of.

The best and final solution would be to change some aspects of the nanojit to allow the assembler to see more of the LIR at once and make bigger level decisions (or hint the processor-specific portions about them), or to expand the LIR opcode set to embrace more specific purpose-oriented opcodes that could hint a code generator like this one about the intended purpose of a section of code. Certainly for loops there is a better way of implementing it in assembly than this. I don't know how likely that is to happen, however.

The plan for the nanojit then is this: it will come out as part of beta 9, turned on by default (you can turn it off in about:config). It has been tested mostly on G5 and G4/7450, though I also ran it on a G4/7400 and it seemed okay. I don't have a G3 running Tiger to test this with, so you guys will be on your own. See if it's faster for your workload. See if it's something you can improve upon. It's a start.

Got it == flaunt it

2010-12-29T18:31:00.000-08:00

It's December 29, and I'm still getting Christmas gifts! Well, I had to brag a little ...

Mozilla sent out their Firefox 4 Beta Team T-shirts today, with a thank-you card that says, "thanks for all your hard work! You rock!" Now, maybe that was for OverbiteFF, or Classilla, or being a pest on Bugzilla, and not your favourite and my favourite POWER ISA legacy Mac browser, but hey! free T-shirt! :-D

The nanojit has been up and running for a few days on my personal dev build and is pretty stable, but the results are currently mixed, and I'm going to do a full post on that when I have some hard numbers to present. The long and short of it is, for many tasks the nanojit is a screamer, but for certain tasks it is actually slower (because remember, the trace and emit step has overhead of its own, and any savings the JIT makes seem to be outweighed by that overhead under certain conditions). The argument is which benchmark to believe, but I'm still working on optimizations to the nanojit, it still seems to be an overall win, and even modulo my author bias I wouldn't go back to TenFourFox without it. Still, it's instructive to note how much optimization we take for granted out of our compilers and how much work we have to do to hand-optimize assembly, and hand-optimizing PowerPC assembly language is in fact exactly what I'm doing. Once I get to a stopping point, we'll observe what Mozilla does with beta 9 while I push Classilla 9.2.2 out the door since we're already tardy on its deadline.

Enter the nanojit

2010-12-21T19:50:00.000-08:00

One of Mozilla's marquee features in Firefox 4 is ever-faster JavaScript. Of course, the real speed gains are only to be had on their officially supported architectures, which now only includes x86, x86_64 and ARM, because of modern JavaScript interpreters' heavy dependence on just-in-time compilation. (See also Google's work on this for Chrome, which naturally is also Intel and ARM ISAs only.) Since PowerPC is the old and busted and x86 is the old yet new hotness, x86 gets the goodies. This doesn't mean that the pure-C++ version of Fx 4's JavaScript interpreter is a slouch (and Fx 4 is faster than Fx 3.6, plus our own compiler juicing, of course), and there is JIT support in Firefox for other architectures such as SPARC even though Mozilla does not consider SPARC a tier 1 platform. Which brings us to PPC JIT.

Some terminology: the pure C (and now C++) JavaScript interpreter is properly called SpiderMonkey and descends directly from Brendan Eich's work when he was at Netscape. All Mozilla derivatives use it; Classilla uses the last C version which was in Firefox 3.0.19, and so does Camino, while SeaMonkey and Firefox use the C++ version which was introduced in 3.5 with TraceMonkey. TraceMonkey's key addition to SpiderMonkey is a clever piece of code called the nanojit, which watches execution through code paths and remembers the "trace" (hence the name). That trace is in fact generated as direct machine code, and run as such, so it is very fast. However, the nanojit is unable to effectively optimize certain types of code, notably code with evals in it and code with many type combinations, and it must run the program to a certain extent to determine what code is "hot" (so very branchy code causes the tracing JIT to give up, because it would use too much memory and CPU time to figure out where the hotness lays). To handle those cases, Mozilla added JägerMonkey to Firefox 4.0, which is a true method JIT, borrowing the assembler that powers it (Nitro) from ~~the enemy~~ WebKit. Naturally, it is also limited to the currently supported architectures.

So that's our year in review; now for the diplomatic cables. If you look in the nanojit, you see tons of files for lots of architectures, including x86 (of course), ARM, SPARC, even MIPS, and of course PPC. Great, you say, the nanojit in Firefox seems to generate PowerPC code. Then you look at the configuration scripts and your heart sinks:

PowerPC nanojit doesn't get compiled or included in Firefox 3.5 or 3.6.

But wait, I hear you cry! The code is there, right? What's NativePPC.cpp for? Well, it's truly part of the nanojit, but it is incomplete. While it works as part of Tamarin, which is Adobe's implementation of ActionScript (and is indeed part of PPC Flash Player), it does not support enough instructions to work as part of Firefox or in fact any Gecko browser, and this is why your red-hot quad 2.5GHz G5 still benches slower on SunSpider than a measly Pentium 4: the P4 gets native code, we don't.

Well, that's about to change. With big thanks to Edwin Smith at Adobe who put up with my constant queries, I'm proud to announce that we're making progress on doing the obvious: expanding the PowerPC nanojit so it can be used as part of TenFourFox. Ed helped me out with some of the fiddly aspects, and we're now full speed ahead on implementing the LIR instructions needed to fully support the length and breadth of JavaScript that the Mozilla framework requires. Already it is passing most of the LIR assembler tests, and when it passes all of the supported ones, I'll start work on integrating it with the main browser. At the end of this, we'll pass our work back to Mozilla so it can be a part of Firefox, if they want it.

When will you get to play with the new hotness, and how much faster will it get? Conservatively, you can expect up to 2-3x improvement on many JavaScript benchmarks, possibly more. This is still not enough to reach x86 parity because we don't implement the method JIT (yet: more on that later when that work actually commences, if it's possible -- TraceMonkey depends heavily on Nitro, so we'll have to see how feasible such a port would actually be). However, it is still dramatically faster and wrings that much more performance out of our beloved machines. More to the point, it will work on any supported TenFourFox processor architecture.

However, when you'll get it is another matter. While this might be ready for beta 9 or 10, such a massive sea change requires heavy testing and may have subtle bugs, so it may sit 4.0 out and appear in 4.1, or it may appear in a later beta. I'll post more about this later when I have enough testing data to decide, but I've got enough working code already that I couldn't resist a little tease for y'all. Great things are afoot!

Changesets available

2010-12-17T12:24:00.000-08:00

Just a brief note that changesets are now available for beta 8, along with up-to-date building instructions, for those interested in hacking on 10.4Fx. Don't let the file size intimidate you; much of that is binary content from the rebranding, although there are some additional bustage fixes that were landed after beta 7.

I am currently reviewing the beta 9 blockers to see if we will release a beta 9, or wait for beta 10 -- each release is about 64MB off our Google Code quota, so I'd rather not blow archival space on a beta release we might not get much benefit from. I'll post a decision here when I've committed one way or the other.

Beta 8 now available

2010-12-15T21:24:00.000-08:00

Beta 8 is now available! Go get it! Like before, we release slightly ahead of Mozilla, so it will still appear as "b8pre" in the version string.

Mozilla has made some interesting changes to beta 8, the most obvious being the redesigned tabs (with a nice rounded edge) and the new tab-modal dialogue boxes. In beta 7 (and any prior version of Mozilla), that link would give you a drop-down dialogue; in beta 8 ... well, just try it and see. The UI improvement is obvious in that a JavaScript application can't lock up your session by throwing lots of dialogue boxes at you, but there are still some questions about performance and user interaction, so you can expect this to continue to evolve. There are also some under-the-hood changes to JavaScript that should improve stability and improvements to fonts.

Specific to 10.4Fx, this version now offers four separate builds: G3, G4/7400 and G4/7450, and G5. The G5 build is the most unusual since the source is very unstable built for 64-bit PowerPC, so simply -mcpu=G5 or -fast will not work. Instead, this build actually builds for G4/7450, and simply turns on the extra G5 instructions, while including -mtune=G5 which does work (effectively making it 32-bit PowerPC with the PPC 970 instructions bolted on, tuned for the G5's instruction dispatch). Together, these tweaks squeeze out about 15% better performance on a G5 than the old unified 7450 version and is steady as a rock. There have also been performance improvements to the G4/7450 version itself, which is now offered specifically for G4e systems.

10.4Fx also adds support for font tables, enabling many of Firefox 4's marquee font capabilities; enables update checking (downloads are still manual, but you now get background notifications when new versions are ready); makes some UI adjustments and fixes all the known 10.4Fx-specific crash bugs. Note that there are still some Mozilla-generic crash bugs out there and this version is still susceptible to them. That said, I've been using beta 8 as my sole browser on my quad G5 and my iBook G4/1.33 for the last couple weeks and it hasn't crashed a single time on either computer.

Have at it!

Beta 8 is go

2010-12-02T21:33:00.001-08:00

Mozilla is planning to release beta 8 sometime next week and so are we. I pulled source from mozilla-central tonight and made the small bustage fixes needed to continue compilation, and this post is being made from beta 8 in debug mode as we speak. There are still a number of blockers that Mozilla must resolve, but I expect that many of them will be deferred to beta 9, especially the Mozilla Sync-related issues. Most of the critical API and stability fixes are in, though, and they look good.

So far in Firefox beta 8 performance continues to get better, there are some interface improvements, and a number of the glitches in the renderer are gone. On our side of the bug fence, update notification appears to work and all of the known crash bugs are stamped out. We're also planning to deploy a G4/7400 version at the request of those users, and I have a few ideas for a G5-specific version. A G5-specific release was always planned, but the Mozilla source is terribly unstable even with -mcpu=G5, let alone -fast, and I couldn't sort this out in time for beta 7. However, I may be able to use ISA-independent tuning parameters to squeeze more oomph out of it, which is useful to me personally typing here on this quad G5. No promises yet but I am planning to do some tests this weekend to see if such a build will be stable.

One noticeable problem which comes late in the game is that unaccelerated graphics performance, which 10.4Fx is intentionally limited to, is slower in Firefox 4 due to a performance regression in the Cairo backend. We really need this, and Mozilla has marked it blocking final as it affects all platforms. The problem is most acute with animated images; you can track this bug yourself in Bugzilla. Hopefully it gets fixed sooner than later. We're monitoring it.

10.4Fx beta 8 also introduces our own infrastructure and livery, just like Classilla. You'll like it.

Watch this space!

Progress on beta 8

2010-11-20T10:28:00.000-08:00

Beta 7 has a rather annoying bug where if you repeatedly open and close the left-click-hold menus on the back and forward buttons, you can make the browser crash (right-clicking is okay). It took a little while to track down the problem, but it is fixed now in the internal tree. This is serious enough that we will insert a beta 8 instead of going to beta 9 as previously planned so that this fix gets rolled out also.

Also, new TenFourFox-specific branding is mostly complete and a new partial livery will accompany beta 8, along with portions of the infrastructure (similar to Classilla's custom user-support infrastructure). There will also be basic support for update checking -- while I do not plan in the near future to duplicate completely Mozilla's automatic update deployment framework, it will check for updates automatically, and facilitate you downloading them manually. Likely we will enable Sparkle-style updates when we leave source parity.

Mozilla is planning to release beta 8 by the end of November and it appears that they will be pretty close to this milestone, so you can expect TenFourFox beta 8 to emerge around the same time.

Picking the torch out of the dumpster

2010-11-11T18:44:00.000-08:00

It seems that 10.4Fx has come out just in time, as Josh Aas from Mozilla reports that the "universal" builds of Firefox 4 beta 7 are now i386/x86_64, not the traditional definition, which is of course i386/ppc.

So this is where the new era begins. Welcome to TenFourFox, where PowerPC lives on. 10.4Fx doesn't support all the features of Fx 4, and may not ever (particularly features like out-of-process-plugins may even be irrelevant on PPC, given that Adobe Flash will be x86-alone any day now). But it does support those of us who still have perfectly good Power Macs, including those of us who still use them for Classic applications, and it includes all the great layout, HTML5 and CSS3 enhancements that are part of Gecko 2.

This blog covers some of the development progress so that you can stay informed. I'll probably be working up a Twitter account next, but new developments will get posted here first.

PowerPC forever!