Long life computing (plus: Quad G5 CPU swapping for dummies)

We are now at the point where the youngest quad G5 is almost eight years old, and TenFourFox works on any Mac that can run 10.4 and has sufficient memory, meaning theoretically even a beige 9500 with a G3 card and a full RAM loadout could be out there (and if there is, I'd love to hear about it) pushing 19 years young.

I'm committed to getting a full useful decade out of my quad G5 systems, and frankly I don't see any reason to stop there. The Quad handles all my tasks except taxes and Android development (I use an Intel mini for Eclipse and TurboTax), including Photoshop, Microsoft Office and HD video; I'd have to rebuy all that just for the dubious privilege of using an Intel Mac. When it gets to the point that TenFourFox's rendering core can't be evolved further, then I'll just throw a Linux box in the server room and run a browser over remote X11. Then I can have my cake and eat it too.

Like me, I imagine most people reading this blog will want to keep their vintage Power Macs operating for a long time as well, especially if you use your Power Mac as your daily driver like me. There are some general things I do with all of my regular use systems (my daily drivers are my Quad G5, a 1GHz iMac G4 and an iBook/G4 12" 1.33GHz, all running 10.4) and these suggestions should help you also. Plus, those of us rocking liquid-cooled G5s are probably starting to notice that the fans are coming on harder, so we're going to talk about servicing the Quad G5 LCS at the end.

Software security. If you're running TenFourFox, you're already ahead of the curve. Congratulations. Here are other security issues to consider with old versions of Mac OS X (as well as our note about OS X's internal SSL).

Save a port, ride an extension. Wait, what? Logic boards and daughterboards are expensive and precious as well as a pain in the rear to fix, and most ports are directly soldered to the logic board, so when a port wears out it can be an expensive and time-consuming swap job. For USB and FireWire, especially for the front ports on my G5 which are constantly being connected and disconnected from my mobile phone and HD video camera, I connected extensions to them and I plug the camera and phone and etc. into the extension cables; I'd rather be replacing a $5 or so extension cable than wrecking my front port daughterboard. Similarly, my Bluetooth dongle connects with its own extension cable (though this is partially so I can find the damn thing).

Rotate your hard drives. Virtually all of our Power Macs are beyond their crib death stage and the next likely failing device will be spinning hard disks. Some of you have solved this problem with solid-state drives, which are a decent if pricey upgrade and certainly immune to mechanical if not electronic failure, but not even the SATA bus in a G5 will get anywhere near the maximum throughput possible; plus, no version of PPC OS X supports TRIM, which means write speed will gradually degenerate. (Linux users might get better benefit.) In my case, I have a stock of SATA I and II spinning drives which I bought new or NOS, and every four or five years I clone the disks (boot off the Tiger DVD and use Disk Utility) to a new set and swap them out. I'm using WD Caviar Blacks which are good solid 7200rpm drives; I did a swap in 2010, my next swap is due for 2015 and I have enough disks in stock to do one more swap in 2020. The swapped out drives are still working, of course, and if the G5 is still trucking in 2025 I might start a more rapid rotation schedule with those old disks depending on how heavily it's being used.

If you're on a system using the older PATA drives, stock up now; NOS stocks are disappearing -- or find something like a Sonnet PCI SATA card, which is bootable and works fine with OS 9 and OS X.

A common question is whether SATA III (6.0Gbps) drives are compatible with G5s and PCI SATA cards, and the answer is, of course, it depends. All Power Mac compatible SATA controllers I have encountered were SATA I (1.5Gbps), including the ones in all the G5s up through the Quad, and SATA II (3.0Gbps) drives work fine with SATA I. If the SATA III drive can be jumpered to run at SATA II speeds, it will work just fine; if it does not have a SATA II jumper, it depends on the drive electronics. Some users have reported that the SATA III Hitachi Deskstars work fine with SATA I, but many SATA III SSDs will not and this can be an expensive thing to explain to vendor support.

Speaking of spare hard disks ...

Have spare RAM, spare cards and spare parts. Consider a body double. Get used to doing basic repair on your machine. Besides hard disks, RAM ages too; both my iMac G4 and my quad G5 blew DIMM sticks around the same time. The good news is that every New World Mac uses industry-standard RAM and I was even able to find brand new PC100 sticks recently for my old Sawtooth G4 file server on Amazon, 14 years after it was taken off the market. RAM is cheap. Have a stock on hand. Beige Mac RAM can often be found from used Mac vendors like MemoryX.

Stock up on PRAM batteries. Later machines like the Power Mac G5 use standard coin cells but earlier G4 systems used 1/2AA 3.6V lithium batteries which may need to be specially ordered.

Another frequent consumable is video cards on those systems with discrete video and PCI/AGP slots. Fans are often the culprit -- the nVidia 7800GT's fan quit in my G5 a year or so ago when I was out of town and I returned to discover the GPU was at a finger-searing 105 C. Amazingly, it was still working and I put a new CPU cooler on it. I have a stock of a couple spare 7800GTs since they're a decent video card that doesn't require two slots in my G5.

Regular PCI cards die as well, though admittedly not as frequently. One of my PCIe FireWire/USB cards is now refusing to recognize FireWire devices plugged into it. Time to swap that out; it's just not worth doing any sort of component level repair on these things anymore. I have a couple in stock too. Sonnet still sells many of these peripheral cards new.

Certain machines have notoriously high frequency failures in certain components. Besides the issues with many early Power Mac G5 models, which I'll talk about in a moment, the MDD G4s are infamous for having iffy power supplies. Since my MDD is my OS 9 workstation of choice, I have three MDD power supplies in stock (I've already had to replace the power supply twice on it since I bought it new in 2002, not counting the power supply replacement under Apple's recall).

If you are unlucky to have a model with a history of logic board failures, take heart in that logic board death is usually an early finding and most of the surviving Power Macs will have a better track record. However, it's often better to just keep an entire body double on hand for your mission critical systems -- a(n as much as possibly) identically configured system, waiting in storage ready to swap in when you need it. This gets you spares for everything, even the logic board, power supply and case; if you have a major electronics failure, just move everything to the body double and work on the original. I kept a spare Apple Network Server 700 in the closet to replace my Apple Network Server 500, which it had to do for a period of a few weeks while I debugged a hardware issue, and my body double quad G5 is now my current quad and the original one is now the spare (with a new CPU and LCS). I also have body doubles for my iBook, my iMac G4 and my Sawtooth G4 and MDD G4.

Prices on Power Macs are starting to bottom out as they become rarer and more interesting as collector's items. This may well be the last chance to stock up on them at a decent cost before it becomes a seller's market.

Have a strategy to recover from catastrophic failure. Besides my body double and spare disks, I have bootable Power Mac compatible discs for Alsoft DiskWarrior, Prosoft Data Rescue 3 and of course retail Tiger 10.4.6, which will boot any compatible Power Mac including the Quad G5. Alsoft even still sells a DiskWarrior for OS 9 systems, and Data Rescue 3 still works with G4 and G5 computers. Support companies that still support the Power Mac; please don't pirate. I'm not an affiliate of either company, merely a satisfied customer.

Throttle your CPU: heat (and heat caused by higher power consumption) is the enemy. Heat diminishes the lifetime of components. Heat dries up thermal compound on the heatsinks. Run your Mac in reduced mode if at all possible, particularly for G5 systems which run comparatively hot to begin with -- my G5 is usually in Reduced mode, and I only flip it to Highest if I have a task I need done quicker or a slow-to-render web page. A tool like Fast and Slow (free) will do nicely; I like CPU Speed Menu, which is a little more configurable, but it's shareware and it doesn't seem its author supports it anymore.

Power Mac G5 Special Section: Swapping the Quad G5 CPU

I have long observed that there are only two G5 computers worth owning: the air-cooled PCI dual processor "DP" 2.3GHz (the later dual core "DC" PCIe 2.3 is also air-cooled, and about 5% faster depending on the benchmark, but has comparatively lower reliability scores), and the liquid-cooled Quad G5, the fastest Power Mac ever made and the one with the best liquid cooling system. The iMac G5s are a big bag of hurt to work on, the early G5s are not substantially faster than the G4s they replaced, and the others have logic board, power supply or liquid cooling leakage issues.

Now, again, by this time in the Power Mac's lifespan most early failures have already wound up in the landfill. Similarly, most of the liquid cooling systems that were going to leak probably already have. However, liquid cooling systems, like the automotive radiators the G5 systems are based on, wear out. There are small evapourative losses from the hoses carrying the coolant, which in these systems are water-based, and thermal paste also slowly loses its moisture and starts to crumble. The very last Quad G5 to roll off the production line is almost eight years old, and if it's still on its original CPU and heat sink it is very likely it's suffering from both these problems.

At this point we should probably talk about what thermal calibration actually involves. All G5 (a/k/a PowerPC 970) systems use fan software controlled by the operating system which needs to know how effective the current cooling system is. According to IBM's technical documentation on the 970, each CPU has temperature sensors and a calibration ROM which stores data about the sensors' set points and fan speeds. Thermal calibration does not actually calibrate the sensors; thermal calibration data for the on-chip thermal diodes is burned in at the factory (IBM calls this information part of the "fuse code data") and if these sensors are bad or incorrect the only solution is to replace the CPU. Instead, what it does is try to find the fan (and, on LCSes, pump) RPM ranges that will keep the CPU in a normal temperature range even under maximal power draw, and writes this data to the calibration ROM. (This is why losing your PRAM battery doesn't screw up your fans. By the way, the PMU/SMU, contrary to popular belief, does not manage the fans either. Resetting the PMU/SMU to fix a fan problem won't help and may scramble the PMU/SMU.)

The upshot is that whenever you do anything to change the cooling characteristics of the system, such as replacing or servicing the CPUs, you need to run thermal calibration again to make sure that the computer knows which RPMs and pump speeds will work. You can't "thermal calibrate" your way out of a failing CPU assembly -- the assembly either passes or it doesn't, and on some earlier versions of the calibration tool you ran the risk of corrupting the calibration ROM if you ran it repeatedly on an iffy unit. Thermal calibration is run from the Open Firmware diagnostic tools on the Apple Service Diagnostic disc. Most people will want to use 2.6.3 with both these models of G5, and the Quad requires it, but 2.5.8 will also work fine with the DP 2.3. To avoid Apple's wrath I won't link these disc images here, but they are not difficult to find. The ambient temperature in the room when you run calibration must be below 77 F (25 C).

A utility like Temperature Monitor or iStat Pro will be useful to know if you're getting close to the edge. I threw together a little tool to monitor your G5 temperature and (max) fan or pump speed in the menu bar. (On my G4 laptop, it didn't install in the menu bar correctly and I don't know why. If it doesn't appear to work, make sure the monitoring process isn't running alone in the background soaking up CPU cycles. I'll look into this later.) The menu bar tool displays the highest temp of the CPUs and the highest RPM of the non-fixed pumps or fans; you can also use it as a command line tool (see the page for how). On single pump LCSes, the same speed is reported twice.

At least on the Quad, ordinarily the intakes idle at 970rpm (ahaha), the exhausts and drive bay fan at 1000rpm, the backside fans at 1100rpm and the PCI slots at apparently a fixed 1560rpm. The LCS pumps, for their part, idle at 1250rpm. As heat increases the CPU fan speeds will increase, as well as the liquid cooling system pumps. Under load the fastest RPM in the system is usually the LCS pumps which are rev-limited by the computer to 3600rpm.

At idle, the CPUs should be below 50 C, especially in Reduced power mode, and the CPUs should be roughly within 10 C of each other or less. (In fact, earlier this spring, my G5 observed a 35 C core, the lowest it's been in years.) The CPUs should not exceed 70 C even under load on any core. If your Quad G5 is seeing core temperatures over 70 C even when the pump speed is pegged at 3600rpm, your system can no longer cool effectively and it's time to service the processor assembly. Resist the temptation to run thermal calibration because it probably won't help you for long if it helps you at all. Switch your G5 into Reduced mode if it wasn't already; assuming you're not much over the 70 C mark, you've still got some time to get all your parts together.

Servicing the DP 2.3 is a matter of unmounting the heatsink, applying new thermal compound (Shin-Etsu X23-7783D is the typical recommendation, though others will work) and putting the CPUs back. Fortunately, most DP 2.3 systems if well-taken care of won't even need that; do so only if your Mac has random sudden sleeping episodes or you see complaints about being "overtemp" in your OS X system log.

The Quad is a different story -- the easiest way, and the only way Apple ever allowed, is to just find a new CPU and heatsink assembly. There is a lot of confusion over which will work and what LCS the Quads came with. Most systems I've encountered have a so-called "version 1" Delphi assembly with a single pump. It looks like this:

For a period of time Apple used a so-called "version 2" manufactured by Cooligy with twin Delphi pumps; despite the apparently later revision the "version 2" LCS actually has the poorer track record. The Apple service manual for the Quad does not say anything about requiring a special logic board for the version 2 LCS, and they are comparatively unusual. I will not discuss them further here. Although Apple used Panasonic LCSes on some older G5s, they never used them on the Quads (even I got this wrong until I did my research). There appears to be no problem with replacing a version 2 LCS with a version 1 LCS; in fact, a version 1 may be the only one you can get anymore.

The version 1 CPU and heatsink should come as an item. If you are buying from an Apple Certified reseller, this is the only way Apple ever sold them. Do not buy them separately unless you know what you're doing, or you're totally crazy; you will have to reassemble them. The part number for the combined CPU and heatsink is 661-3729. When shopping around for these, don't pay more than a couple hundred dollars -- I now have three spares, and the most I spent for any one of them was $135. The magic words you want to hear from the reseller are "Apple Certified Refurbished" (or, less optimally, "passes thermal calibration"). If you don't hear these phrases from them, do not buy. Prefer the Certified Refurbished parts, because these came from Apple with new thermal paste and new coolant and they'll last for years more if you treat them right.

(If you can't buy a decent used CPU, all is not lost. There are ways to service the liquid cooling system and the CPUs by hand if you're willing to do a bit of grunt work and are comfortable with draining and replacing the coolant manually. Heikki Lindholm offered many great tips while I was writing this and I direct you to his excellent treatises on servicing the Quad G5 cooling system and pump assemblies.)

In addition to the CPU, you will also need:

• An X-acto knife or small flathead screwdriver to use as a pry tool.
• A Philips 0 screwdriver.
• A 3mm long-handle flathead hex driver (I used an Eklind part# 54930).
• A 4mm long-handle ballhead hex driver (I used a Bondhus part# 13160). Both are available from Amazon and most special-order tool retailers. The tool shaft needs to be at least 8" long; shorter ones will not be able to get to the screws.
• A grabber of some sort to go chasing screws; you can use a magnetic one as long as you keep away from the hard disks.
• A copy of the ASD for your G5; burn it to disc and have it handy (remember that ASD 2.6.3 requires a DVD).

Power down the Mac and unplug it. Let it cool down: not only is the radiator potentially hot enough to burn you, but working on the G5 logic board when it's hot may damage the logic board. Flip the lever on the back, remove the side panel and remove the air deflector, and put them aside. Pull the front fans straight out, in front of the radiator.

The processors are covered by a heat shield with a G5 logo. If you've never serviced the CPUs before (or Apple serviced them), this shield is locked in place by a little plastic pin on top. Pry this very tenacious pin out with the X-acto knife or small flathead screwdriver verrrrrry carefully so that you don't go damaging anything else. This will destroy the pin most likely, but all our Macs are out of warranty and you may need to do stuff with the processor again in the future. The heat shield will now become very loose and mobile. Shift the shield to the left and remove it. In front of the front radiator is a little plastic intake frame; rotate it clockwise from its mooring around the radiator so it can come free from its standoffs and pull it out.

Now it's time to get the old CPU assembly out. Put the computer flat on its side. If you don't do this, the screws will fall out when you remove them and end up in the power supply compartment, and that really sucks! Follow Apple's steps exactly and loosen/remove the screws exactly in numerical order (note: the captive float plate screws are not really captive and can come out, so be careful):

(The ballhead screws in the last step may have to be accessed at an angle; that's why they're ballheads and not flat.)

Once the screws are loosened and/or removed as directed, disconnect the black fan control linkage by pulling the two halves apart, and grab the centre of the metal frame. The CPU should lift straight up and out with very little force:

This is what you should see with the CPU out. Yes, it's dusty. Clean the dust out and in particular make sure there are no dust bunnies floating over where the CPUs attach (those arrays of little square pins).

It is particularly critical not to lose any screws: in particular, the screws attaching the CPUs to the logic board are actually an integral part of the power supply connection to the processors and the computer may not be able to even power up without them.

If the screws despite your best effort do get into the power supply compartment (and don't ask me how I have experience with this), you'll need to give them an escape path. Pull the CPU out and also remove the hard drives because you're going to have to shake the computer around. Unmoor the rear exhaust fan assembly by pressing in the tabs facing you so it can move freely and won't block any flying screws (you don't have to disconnect it unless you want to). Pry up the adhesive pad on the bottom of the partition between the CPUs and the power supply (just enough to get to the screws; don't toss it) and undo both Philips-head screws to remove the partition. Unplug the power cable near the PRAM battery and thread it free as you lift the partition. The partition is flexible and you may need to bend it a bit to get it out. Turn the G5 upside down and remove one or both of the screws on the underside. This will loosen the power supply so the screws can escape. Now it's time for calisthenics with your Quad:

Get all of the screws out! Do not leave any of them loose in the machine! I had to literally put my Quad over my head and shake it to get them loose enough to fish for with the grab. Once you've got them all back, replace the screws in the underside, put the partition back (rethreading the power connector through and plugging it back into the logic board), put the screws back in the partition, replace the adhesive pad and put the rear fan assembly back into its original position, making sure the tabs lock into their little slots.

Now get the new CPU straight up out of its box, lifting it up by its metal bridge (it should be stored such that the heatsink is on top, which uses gravity to maintain the seal between the heatsink and the actual processors). Do not lift it up by the radiators or heat pipes or you will bend them and reduce the assembly's cooling performance. Used assemblies may or may not come with caps over the processor connectors; remove them if they are present, and loosen the float plate screws on the new CPU assembly like you did in the first step of removing the old CPU. Next, put the screws in their holes before installing the assembly: it's much harder to put the screws in when the CPU is already in the case. Carefully align the CPU over the standoffs and put it straight down into the computer:

The CPU should go in nice and level. Press down on the bridge only to make sure the CPU is seated; don't press anywhere else. It should give slightly with a little thunk. Do not apply significant force or you will damage the CPU, the logic board or both.

With the CPU now in place, tighten the four captive ballhead screws (the ones in the last step you did when loosening the screws before pulling the old CPU out), going in order from top to bottom, then middle top and middle bottom. Do not overtighten -- tighten only up to mild resistance. If the ballhead screws are not engaging well, the CPU is probably not seated correctly: make sure the float screws are loose, carefully pull the CPU out and reseat it. You can also try tightening the top, middle top, bottom and middle bottom in a different order; this may enable the CPU to move around a bit into a more favourable position.

Once the ballhead screws are engaged and gently tightened:

• Tighten the remainder of the screws in the reverse order you loosened and/or removed them, finishing with tightening the captive float plate screws. Again: do not overtighten.
• Connect the black fan control cable.
• Put back the plastic radiator intake frame; guide its rear tab to the back around the processor standoffs and rotate it so that its clips grip the front radiator.
• Replace the heat shield only if you want to.
• Replace the front fans and put back the clear plastic deflector.

Connect your G5 and plug it in, cross your fingers, and hold down Opt-Cmd-O-F while turning it on from the power switch. If you did this right, one of the front red LEDs will briefly light and then turn off, the G5 will bong and you will enter the Open Firmware prompt. (If you don't get a bong, the CPU is not in place correctly or, horrors, is damaged. Loosen it and remove it. See if it will boot with the old CPU.) Release the keys, type eject cd to open the optical drive, put your ASD disk in, and close the optical drive. On my ASD 2.6.3 disk, I had to type boot cd:,\diags (yes, you need the colon, the comma and the backslash) to start the diagnostic program, the only time your G5 will ever "appear" to boot Mac OS 9. :) The fans will come on full blast; this is normal and the fans will remain loud during the entire diagnostic process.

After the diagnostic program finishes loading, select Thermal Calibration from the pull down menu. Make sure the room is below 77 F/25 C and that the front of the machine is not facing any heat source or heat exhaust. Once you begin thermal calibration on the new CPU assembly and it succeeds, you may cause issues with your Mac if you put the old failing CPU assembly back in later. This is the point of no return. If you're lucky, this is the result:

Congratulations. Now put the side panel back on, run the rest of the self-tests and keep on trucking.

(And marvel about how quiet it is now when you reboot into OS X.)

Meanwhile, I'm pulling 29 beta to transfer the changesets over for a TenFourFox 29 unstable release. Still a couple of moderate issues to fix, but it's getting there and is mostly in a state I think you'll be able to play with. More about that probably next week.

SS'more SSL SSuckage

Today's nasty flaw is a critical weakness in certain versions of OpenSSL, which can expose private keys and credentials and apparently has been part of OpenSSL for at least two years.

Although Mac OS X's built-in SSL library is based on OpenSSL, no version that shipped with any version of Mac OS X (even 10.9) is vulnerable to this particular bug; the issue only exists in OpenSSL 1.0.1 through 1.0.1f and 10.4-10.9 are based on either 0.9.7 or 0.9.8. However, this could be a problem for our Linux users, and an application that ships with an updated version of OpenSSL (or MacPorts, Fink or Tigerbrew/Homebrew users who built their own OpenSSL) could also be vulnerable. And, of course, there may be other issues with OS X's built-in SSL library that have not been patched either.

TenFourFox is not vulnerable to this problem directly because we use NSS, not OpenSSL, which is a very different SSL library. Unfortunately, the attack method is straightforward and does not appear to leave any trace, so it's entirely possible for a service you use to have been victimized already without their knowledge. If that's the case, an attacker can masquerade as that site and present an entirely legitimate certificate which any browser will accept, since it's signed with a perfectly valid private key. We really have no idea how deep this rabbit hole goes.

Tiger time ... was six hours ago? (updated for 10.5)

One of our more interesting trouble tickets was one where the user's incorrectly set clock caused SSL certificate validation to fail (they kept appearing expired), leading TenFourFox to refuse to load other items such as style sheets, concluding they were also insecure.

The root cause of this problem is that 10.4 systems have an annoying tendency to lose synchronization with an NTP timesource (NTP being the typical protocol used by network time servers). This is a problem Apple has acknowledged, and has a service article on with an official remedy (essentially, to put iburst into /etc/ntp.conf). Unfortunately, this approach isn't a complete solution: while it does reduce initial negotiation time, systems like mine (and the user's) with long uptimes without sleep or shutdown will still eventually go out of sync.

UPDATE: There is apparently some argument over whether this was really fixed in 10.5. The good news is that the approach below should work for any version of Mac OS X with ntp.

As the Apple article points out, the first and easiest way to reestablish the connection is either by rebooting, or unchecking and then re-checking the time server checkbox in System Preferences. If your system is powered on and off regularly you may not have a need even to do this much, though you may want to make the iburst change to make any time change occur more quickly when your computer restarts. Similarly, if your computer is only intermittently connected to a network, you'll be reestablishing the connection periodically anyway. My iBook G4 falls into this category, so I left it alone.

On the other hand, my quad G5 and my Sawtooth G4 file server are always running and always connected to a network, and I have a local NTP timesource they should be syncing to. For these systems, I actually turned off syncing to a timeserver in System Preferences. Instead, in Terminal.app, I sudo tcsh and crontab -e. This opens a root shell (which sudo should ask you for a password for, right?) and edits the regularly scheduled cron task table. To the bottom of this file, add

2 * * * * /usr/sbin/ntpdate -s time.apple.com

Save the file in your editor (crontab should confirm that you changed root's cron job table). Every hour, at two minutes past the hour, this line will query the Apple time server and force your computer's clock to immediately sync to it. If you use a local timesource, substitute its hostname for time.apple.com. You might want to immediately ntpdate -s time.apple.com after editing the crontab to make the clock immediately correct.

Because it uses a "manual" method, this approach doesn't require a constantly open socket to the timeserver and makes changes instantaneous to boot. Checking even less frequently than hourly will probably work too, but you can experiment with that yourself.

Properly maintaining your computer's clock is a necessary component of security because SSL certificates and access tokens and credentials are always timelimited. Worse than having a clock set such that certificates appear expired is to have your clock set such that bogus expired and possibly previously exploited credentials appear valid. Make sure that Tiger time is the right time.

The G5 service article is next.

And now for something completely different: Brendan Eich, and FirefoxOS review for Power Mac users

I've been writing up my FirefoxOS review for this blog/audience, but the flap over Brendan Eich (briefly) as Mozilla CEO eventually came to a crashing halt today with his resignation, not only from the CEO post but from the board and the entire organization. Besides a few private inquiries I got about TenFourFox's use of Mozilla technologies, this news sort of overshadows the entire thing. So here goes. If you want to skip this part, scroll down to the review.

UPDATE: In his own words, leaving Mozilla.

We have many users across the entire spectrum on gay marriage. (That's an important point I'm going to come back to.) This blog is apolitical -- I am intentionally expressing no personal viewpoint on this subject, because it's irrelevant. My use of Mozilla and Gecko, and my actions as a Mozilla contributor and security board member, is because I believe it to be the most community-driven engine, the most standards-friendly engine, and the most open engine. It's not based on support or opposition to the views of any one person in Mozilla, even the CTO (when he was the CTO) or the CEO (when he was the CEO).

For those who don't know the history yet, California (I live in Southern California) qualified Proposition 8 for the ballot in 2008, which stated that the following text, almost in its entirety, be added to the state Constitution: "Only marriage between a man and a woman is valid or recognized in California." This was against a context of several long court battles going back for at least a decade, along with the earlier Proposition 22 in 2000 which passed but did not survive subsequent legal challenge; Prop 8 was designed to address the constitutional issues where Prop 22 had failed. Prop 8 survived to reach a vote and passed, 52.24% in favour (see the Wikipedia map for breakdown by county), and was immediately challenged in court. The California Supreme Court upheld it in 2009, but the United States District Court for the Northern District of California overturned it in 2010, affirmed by the Ninth Circuit Court of Appeals. In 2013, the U.S. Supreme Court denied legal standing to those attempting to defend it, technically overturning the Ninth Court, but leaving the original ruling by the District Court intact. Gay marriage has been legal again in California since 28 June 2013.

California also has a law requiring disclosure of certain personal information of individuals who donate $100 or more in support of or opposition to a ballot measure. This information is public, and is downloadable from the California Secretary of State. Among other groups and news outlets, the Los Angeles Times publicized the donor list for Proposition 8 shortly after the measure passed, making it searchable; soon the entire donor list was widely circulating as boycott targets.

One of those names was one Brendan Eich, of Mozilla, who had donated $1,000. Eich started at Silicon Graphics, where he worked on IRIX, and eventually made his way to Netscape Communications in 1995 where he became the inventor of JavaScript (for Netscape Navigator). After America Online bought out Netscape in 1998 he helped to spin it out as Mozilla, which later became independent, and where he remained until now as the CTO since 2005. Eich's contribution was public record for some time, but did not come to light until 2012 when it was rediscovered. Many of Eich's coworkers, including some of his closest, were shocked and dismayed. However, virtually everyone, even his detractors, agreed that he had never made his views public at work and had left his personal feelings at the door. Such was the background when he was made CEO by the Mozilla board, apparently against the wishes of three board members who later resigned, preferring an external candidate. The reaction was swift, including at least two high-profile boycotts, an employee request for him to resign, and many prolonged flamefests in every corner of the Internet. Eich was repeatedly asked to recant his views and, most recently given an opportunity in an interview with the Guardian, he did not.

Today he did step down, apparently for good, not even returning to his previous position as CTO; it seems his days at Mozilla are done. It's not clear where he's going next.

Free speech in the United States is heavily protected, possibly to a point that our readers from other countries will find absurd, but it is and has been for decades. However, those protections extend to preventing government from abridging those rights; private actors are not so enjoined. One interpretation is that this sad series of events is the system working exactly as designed: Eich had a right to have those views and express them, and those opposed to those views had a right to express their opposition and their displeasure with his employer. That is, essentially, exactly what happened -- he has a right to free speech, but not to be absolved of the consequences, which in this case was likely a "fall on your sword" resignation.

I agree with this principle, but only up to a point. There are clearly certain views that are now so far out of the mainstream that no one will tolerate them in a high profile position anywhere. You may still get to be a racist, or sexist, or express your dismay over miscegenation, in certain parochial environs but at large the vast majority of Americans do indeed reject these views as antiquated, wrong and damaging. Certainly any CEO as high-profile as this would expect to be hounded out for holding such opinions today. However, generational change happens generationally. We did not expect many of the most ardent racists to reject their views; some may have moderated them, and some made public apologies, but many went to their graves with the same beliefs and after racial integration laws and other marked civil rights changes it was their children who grew up in a different world. Gay marriage has not reached that point, and is qualitatively different in that at least a large portion of several religious traditions continues to regard it as immoral. As recently as 2010, only 44% of U.S. citizens supported it; President Obama himself did not reverse his previous statement on gay marriage until 2012. As of 2013, a slim majority supports it, which means almost half of the country still does not or is undecided.

Opposition to gay marriage, despite people's feelings and protestations, is still a mainstream view in the United States (to borrow Lt. Gov. Gavin Newsom's infamous phrase, "whether you like it or not"). It is a mainstream view that is likely to erode bit by bit because of recent Supreme Court rulings on the federal Defense of Marriage Act, which struck it down and will likely be a precedent to overturn laws and constitutional amendments against gay marriage in those states that already have them, and the children of people today who are against gay marriage will grow up in that world instead where it is legal and in many places accepted as normal. But this is generational change. Until then, a significant proportion of Americans hold these views, and many will hold them until the end. In fact, if there were indeed a religious motivation, would we expect differently? More to the point, is anyone with a sincerely held moral objection to gay marriage a "homophobe," even if they accept the legality of gay marriage as reasonable or at least unopposable?

It is highly corrosive to society to exclude or relentlessly ostracise those people whose views were thoroughly acceptable just a short while ago -- it cuts out large swathes of people while accomplishing nothing practically, especially now that the legal argument against gay marriage has been made all but void by the Supreme Court, and acts as a chilling effect on free discourse. There is no honour in establishing a blacklist even if you are on the right side of history. Ah, you say, but Eich's actions are beyond simply expressing a view; he acted willfully with this campaign contribution (and arguably others) to restrict the rights of gays. To this assertion I say you'd better be careful. The U.S. Supreme Court has repeatedly ruled that campaign contributions are indeed a form of free speech like any other, a view doubled down upon just today by McCutcheon v FEC. Like it or loathe it, this was his part of expressing his view in the democratic process just as you or I get to do, and if you don't like this ruling then you should remember it was also this same Supreme Court that overturned DOMA. Live by the sword, die by the sword.

I've had my disagreements with Eich as a community member, but they always have been resolved amicably even when things didn't go my way, and while my interactions with him have always been of a purely professional nature he was always professional. In the end this whole sad affair might be perceived as decisively striking a blow for gay rights, but no one really won here. Certainly not Eich -- where will he go now? Even if he had returned as CTO, who knows if this would have blown over this time? He'll always be "that guy," and someone will always want his head; some posts don't believe he's fit anymore for any job in the tech community. Definitely not Mozilla -- they should have foreseen this and predicted how bad a message this would send to their community and employees and never made him CEO in the first place; now one of their biggest carriers of institutional memory and technical knowledge is under a cloud and will likely never be a meaningful contributor to Mozilla again. This was bad business, plain and simple. Their ill-advised move making him CEO looked tone-deaf; now they look diffident as well. They'll suffer the public relations ramifications of this whole story for years.

But proponents of gay marriage have also lost. A vigourous mob took a man who by all accounts never brought his beliefs to work and who embodied the Open Internet principles on which Mozilla is still grounded, and hounded him out of a job. (Yes, I know he "resigned." Everyone in that position "resigns." Many are told to.) This makes martyrs out of whole cloth and hardens the viewpoints of the people you need to accomplish your goal. It represents the movement badly, and it sets back the generational change that is desired.

Adlai Stevenson once observed that a free society is one where it is safe to be unpopular. If you believe this sort of mob justice is acceptable, pray it never sets its sights on you.

Done with that topic. Now that some of you have stopped reading in fury, let's talk about Firefox OS and the Geeksphone Revolution Revulsion. In this post-PC age, Mozilla has virtually no presence on mobile devices and tablets, and even though there is Firefox Android it's large and unwieldy and hardly used. So Mozilla has focused on the feature-phone market to bring low-spec devices that may not run Android well into smartphone territory. That focus brought us FirefoxOS.

FirefoxOS has direct benefit to us in TenFourFox-land because it is written for (what are now) low-spec phones somewhat straddling the gap between feature phones and smartphones. Happily, these phones have similar specifications to our aging Power Macs, and since the rendering is all cross-platform code, we can reap these benefits on our own hardware. But let's say that you're tired of the Google-Apple binary in smartphones (Windows Phone? really?). You don't trust any of them, not even the new and sexy Microsoft, and you're tired of having to void your warranty to run something else. Ostensibly, at least, you have a third choice and this is it.

My previous review of the Revolution Revulsion was singularly disenchanted with the poky Atom CPU, the crummy screen and the plasticky feel. Most importantly I was really p*ssed off that it came with Android, and AOSP Android 4.2 at that, not the Firefox OS phone I had ordered despite Geeksphone's advertisements to the contrary. Those criticisms still stand.

Fortunately, no thanks to Geeksphone, I discovered where you install it. In Settings, you can select another operating system, and Firefox OS is one of them. It does not dual boot -- this will flash your phone. It will automatically download a build of FirefoxOS from Geeksphone's servers and reboot:

And here we are at the main screen of what appear to be a completely vanilla install:

FirefoxOS distinguishes itself with its particular brand of "apps" which can be hybrid offline or online, but 100% written in HTML5. This isn't as bad as it sounds considering how functional HTML5 has become. Even things like the dialer are HTML5, interfacing through privileged DOM objects with the phone hardware. A number of them come with the default install, but you can delete these freely. If we swipe to the right side, taking a screenshot (Power-Home), we see the rest of the built-in apps:

These are very basic, but functional (except for Mail, which for some reason I could not persuade to work with Gmail). Dialer, SMS, browser, phone book, calendar, music, it's all there. The camera is similarly basic, but here the Revulsion's hardware lets it down. The camera's low light performance is slow,

and the flash white-balances poorly, which ought to be correctable in software. I'm not sure if Geeksphone or Mozilla is to blame here.

The HTML5 apps included with the machine span the gamut; here are the Social ones. (Notice the Spanish labeling.)

More are available in the marketplace, like my current favourite timewaster, 2048:

Happily, notifications and control strip operations work pretty much like most other phones do.

However, Android users such as myself will keep trying to hit the back button, and this doesn't work -- you go "home" and pick other apps. iOS users won't have any problem.

Bluetooth pairing was a little wacky with my G5. It's better if you initiate pairing from the Mac -- if you try to do this from the Geeksphone, then the pairing dance gets a little wild and they try to pair at different intervals and it becomes messy. Once it's done, though, it works fine and transfers files and pictures. You can even queue Bluetooth transmissions up into a big all-at-once blob, which is something my Android Nexus 5 does not do well.

I am also delighted to note that the SD card is fully accessible over USB, so you can dump files and music from your Power Mac right on it. I wish Nexus phones would support this again.

The phone identifies itself as Firefox 28, but gets updates from Geeksphone, not Mozilla.

I haven't yet received an OTA update, so I don't know how well that works. By the way, you should consider the switch to FirefoxOS practically one-way. You can go back to Android, but there is no installer and you will have to reflash the device. But Android is so lousy on this I don't know why anyone would want to.

I don't think FirefoxOS could be my regular phone, at least not yet. I dithered over swapping in my 3G SIM and seeing how long I could put up with it, but decided not to -- I still have a lot of Android apps I like, though I suppose I could run them on a tablet and use a Firefox phone besides, and my work depends on me being accessible at all times and I'm not 100% convinced of that level of reliability with FxOS yet. The biggest problem is, of course, this particular phone: it sucks, even though it sucks a lot less in FirefoxOS than it does in AOSP Android. That does bode well for Mozilla, and a very inexpensive phone running FxOS should be a lot more pleasant than Android on the same hardware plus a lot less restrictive. But I'm not really in that market -- I like high-end hardware, and FxOS is not compelling on that, or at least not on the choices we have now short of buying another Nexus 5 and trying to flash it.

Those of you who are in that market, though, should consider FirefoxOS to be very serviceable. I'd look at another handset than this one, and I wouldn't pay a lot. But if I had to, I think I could use it and adapt to it. It's going to take something better than that to wrest established smartphone users away like me, but for people upgrading to a smartphone from a feature phone, Mozilla has a chance with those people and they might have a chance with you. And being Power Mac friendly, or at least not Power Mac hostile, is a big plus. If Mozilla finds a better hardware partner, I myself would strongly think about buying another Firefox OS device and giving it another shot. Let's hope and see, because I don't want just two tech companies in the mobile space -- and Mozilla needs all the help it can get to stay relevant.

A polite request: no comments about Brendan Eich -- I will delete them, even the ones that agree with this view. I don't have time to moderate a flame war. This post is the only statement I will make on that topic.

TenFourFox 29: sucks less

First, ObTenFourFoxNews. 29 is now substantially less crashy and many of the severe bugs are fixed. Although there will be some visual differences from Australis on 10.6+, mostly for performance reasons, it now looks much the same and themes based on it should just work. Major issues yet to be fixed include webcam support, which is totally broken right now; an issue with new tab thumbnails showing a blue tint which is obviously an endian problem; and doing some more work on missing window controls and traffic light buttons. You can track progress in issue 267. I'm hoping to migrate to the beta changesets and then issue an early beta pretty soon-ish so people can bang on it. No ETA yet but I'm hoping for end of the month. Please note that being unstable, there may still be some known minor to moderate issues that I will defer for later fixing.

One of our more interesting trouble tickets was one where SSL certificates came up bogus and certain style sheets wouldn't load. Eventually the user was able to determine that it was his clock that was to blame -- because the time was wrong, it would fail certificate validation because the SSL certificates appeared to be expired. 10.4 does have an issue staying synchronized to an NTP time source, which I've noticed on my always-on G5 and G4 server, and I do have a fix for my systems that involves a little minor surgery I'll put in a future blog post. However, in a way it's peculiarly reassuring because it means that TenFourFox's certificate validation does fail when it's supposed to.

This is also a good point to publicly thank Chris Trusch for all he does answering questions on our user support Tenderapp site and his work managing the localization team, whom we also greatly appreciate for faithfully translating our unusual strings into all our supported languages. You can help support our user base as well. Feel free to jump into threads if you've got something to add, and if you'd like to actually take a formal support staff role contact me in the comments.

The G5 liquid cooling surgery went fine without major issues, though the grounding screws fell into the power supply side, requiring partial disassembly of the case to get them out (grrr). However, it is now substantially cooler and quieter and passes thermal calibration like a champ. Heikke Lindholm, who maintains probably the definitive Quad G5 LCS servicing document available, and I have been conversing about best practices for keeping G5 systems running and I'll also make a blog post about how to do the LCS exchange if you are able to locate a Apple Certified Refurbished part (for the Quad, it's 661-3729 -- make sure you ask them if it's Apple Certified Refurbished, or at least verify it passes thermal calibration, and don't pay more than a couple hundred dollars). This will generally apply to other LCS G5s, but I'm primarily interested in the Quad, and that's what we will primarily discuss. If you don't want to buy a new LCS unit or can't, Heikke's instructions will help you with doing the work yourself.

The FirefoxOS review is next.

For better or worse: Power Mac G5, meet Australis. (Oh, pick him up off the ground, please, he crashed again.)

After a lot of sweat and screaming this weekend, I am relieved to state that Australis can be coerced to build and start on 10.4, and even do some basic browsing. Meet the very embryonic TenFourFox 29.

Unfortunately, right now it's hideously buggy. Issue 247, a known bug with personas where the traffic light buttons disappear at times, is prominent in every window. I'd assimilate this as a feature except that it sometimes will partially paint them and sometimes not at all, so we'll have to find something that makes it better. The background colour of the window is wrong, too; it always appears inactive, so the tabs look like they're not attached to anything. Icons in the menus look totally screwed up though they appear properly in the main window, and it also has random crashes on some web pages that look like an interaction with the font code we had to hack. But it does build, it does link, and the chances of there being a TenFourFox 31ESR are now better than even money. Once the browser is halfway stable, I'm going to pull the beta and rewrite the changesets. (Ignore the Aurora livery; we don't have special TenFourFox livery for aurora builds since we rarely issue them.)

Linux users, my success is bad news for you, however: this means that bug 961488 which is preventing Firefox from running on Linux/ppc32 is Linux-specific (and possibly affects *BSD as well) because I didn't trigger it in TenFourFox. Hopefully one of you figures it out. I left a place to start looking in that bug.

While I've only used it briefly I'm pleasantly surprised to note that it doesn't perform badly, considering, since this is a debug build and only trivially optimized. It's obviously much more wasteful of screen real estate than 24, but this is what we have to live with, and there are themes to revert it for those of you who already know you won't like it. Personally, I don't like it either (on my 10.6 machine as well) because it smacks of Chrome and it's a bit gauche, but I can live with it, and there are admittedly some nice things about it.

One final note of concern: there is a problem looming on the horizon, and it's another issue with our linker (issue 266); once again, Mozilla has overbloated XUL to the point where it won't link with our tools. The offender appears to be the newly-enabled International Components for Unicode support (which isn't in 24; this doesn't have anything to do with localization per se), which increases XUL by 10-20MB and makes the linker run out of room to link branches together. Mozilla does not install ICU support on Android or FirefoxOS, which is fortunate because therefore they must allow building without it and it's not likely it will be mandatory for Firefox 31ESR either, but we'll have to deal with the problem sooner or later because other future features might bulk the browser up too. If we can't link the browser, we can't build the browser, and we'd have to drop source parity.

Speaking of FirefoxOS, I did get it installed finally on the Geeksphone Revolution, which I have since christened the Revulsion because it sucks. In spite of the lousy hardware, however, FirefoxOS isn't bad at all; I'll have a review blog post up later this week. Still waiting for one last tool to do the service work on the G5, which I will also write up since I know some of you have asked about that too.

Update on 29

After a lot of sweating, JavaScript in TenFourFox 29 now runs with PPCBC once again. There seem to be some performance improvements, too, to the tune of about 5-8% on V8. Not bad. Now to get the rest of the browser up. No ETA on that yet.

The refurbished G5 spare is doing well; even in Highest performance it is fairly quiet by comparison, and in Reduced mode the loudest fan in it is actually the video card's. I'm waiting for tools I have on backorder to remove the dying processor assembly from the original quad G5 and I actually located several Apple-certified Refurbished complete assemblies with heatsinks and fully serviced liquid cooling systems, so I should have some tested-good spares that pass thermal calibration to substitute in (and I won't have to service the older one quite so urgently). I'll have some notes and a little bonus toy to play with in the very near future.