Sunday, December 21, 2014

31.3.1pre test build available

Everyone needs a break, so I took a break from my Master's project to experiment with a few ideas I've been ruminating over (it's also a convenient excuse to goof off). We're always looking for more performance from our older machines and sometimes this requires rewriting or changing assumptions of the code where this can be done without a lot of work.

Multiprocessing/multithreading has been one of those areas. Since around Fx22 Mozilla has been using background finalization for JavaScript (so background finalization was in 24 and 31 but not in 17) which is to say that the procedure for deallocation of objects is not done on the main thread; finalization is a separate, though related, process to garbage collection where finalized objects with no remaining references are reclaimed. This, at least on first blush, would seem like an unmitigated win, especially on machines with multiple CPUs -- you just do the work on another core while you're working on something else. But interestingly it has some very odd interactions on my test systems after long uptimes. Some profiling I did when TenFourFox just seems to be sitting there waiting showed that it wasn't TenFourFox (per se) that was twiddling its thumbs; the OS X kernel was stuck in a wait state as well and it seemed to have been kicked off by ... background finalization, waiting on threading management in the kernel, which had temporarily deadlocked. This may be the source of some unexplained seizing up that a few people have complained about and we had no obvious way to reproduce.

We can't do anything about the kernel (this is another reason why I'm very concerned about how Electrolysis will perform on 10.4/10.5), but background finalization can be defeated with a few lines of easily ported code. This doesn't make the browser faster objectively, mind you; it just "spreads the badness around" so that finalization occurs predictably, and in a manner that makes it more likely to complete, which in turn makes garbage collection more likely to complete (which since the fix in 31.3 now can run on a different core and doesn't appear to trigger this specific issue), which in turn reduces the drag on performance. In effect, this rolls this aspect of JavaScript back to Fx17.

The result is more consistent and smoother, even if it's not truly faster -- especially on the G5 where bouncing around in code generally hobbles performance -- and does not seem to affect uniprocessor systems adversely, but I'd like to get a test out where you can play with it and see. You should not expect much difference immediately when you start; in fact, startup time might even be slower, and memory usage will show little if any improvement. This is just to improve the browser's responsiveness in terms of staying reasonably quick after multiple compartments have been allocated and need to be scanned.

Speaking of, I also implemented the reduction on tab undoes (to 4) and window undoes (to 2), which also reduces the number of compartments that must be scanned (you can override this from about:config, but be aware that every tab and tab-undo-state you keep in memory remains active and so the garbage collector must evaluate it), and I threw in one other change that forces substantial additional buffering of video playback just to see how this works out for you lot. Like the change in finalization, this "spreads the badness" by forcing the browser to build up a large backing store of fully decoded video frames before playback (up to 10 seconds' worth depending on available memory). Videos may appear to stall initially, but then can play more smoothly because decoding is now more aggressively buffered as well, not just downloaded video data. This quad shows improvement only in that playback is more consistent, but on my 1GHz iMac G4 many standard-definition videos on YouTube are now noticeably less like a slideshow with audio. It's possible to overdo this setting, so I've settled on a conservative number that seems to work decently for the test machines here (it's baked into the C++ code, sorry -- you can't twiddle this from about:config). The minimum recommendation for video is still a 1.25GHz G4.

The tab undo and buffering changes will be carried into 38, but 38 introduces generational garbage collection (to us) and I have to do some testing to determine if it will react adversely with GGC's system assumptions. Please note that I only built this for G5 and 7450 mostly because I want testing on a good mix of single and multi-CPU systems (7400 users can use the 7450 build if you really want to, but sorry, G3 folks, you'll have to wait for the next scheduled release), but I did the building on my new external solid state drive which reduces build overhead by as much as 30%. Not bad! I'll post some stuff about the RAM disk and SSD build testing I've been playing with in a future entry.

Downloads available from the usual place.

Saturday, December 20, 2014

Time, time, time, see what's become of ntpd

UPDATE: After digging through codepaths in ntpd, I'm concluding that an outbound connection -- i.e., you connecting to an external time source -- is probably not vulnerable, at least not to the most serious of the flaws below (ctl_putdata() and configure()). By default OS X does not allow inbound connections, which definitely are vulnerable. If you have changed this configuration, however, you should still update, and the sixth flaw in receive() may still apply, although the exploitability of this particular flaw is believed to be highly unlikely. The tools are still available if you want them.

The sec-buzz this time is a package of six, count 'em, six vulnerabilities in OS X's Network Time Protocol daemon, or ntpd (and possibly its associated tools, see below). NTP is one of the oldest Internet protocols still in use, starting with NTPv1 RFCed all the way back in 1985 -- in fact, the very same David L. Mills who wrote the original NTP RFC 958 is still the one maintaining it today, almost twenty years later. NTP is an extremely accurate and somewhat complex means of time synchronization that can keep Internet-connected clock devices and computers tightly coordinated with highly accurate timesources. In theory, NTP can capture time differences as small as 233 picoseconds, or 2-32 seconds; in typical network environments, it can keep accurate time within a few tens of milliseconds (currently the internal NTP server at Floodgap has an average dispersion of about 15ms), and as low as a single millisecond or less on a local network (my NetBSD/cobalt server is showing 0.2ms).

Accurate time is very important, especially for security, and virtually every modern platform offers some manner of time synchronization; almost every Un*x or Un*x-like operating system uses ntpd, the reference implementation (don't start that OpenNTPD crap here, please), including Mac OS X. And no surprises, the version included on every release of OS X is vulnerable up to and including Yosemite which only runs ntp 4.2.6.

The immediate response is, "but wait! wait! I don't run a time server!" No, you don't (probably), but if you have "Set date & time automatically" checked in Date & Time in System Preferences, you're running ntpd. (If you did the change I recommended in an earlier post to switch to intermittent ntpdate in cron, you're not running ntpd but you are still vulnerable to one of the flaws; read on.) If the remote server sends you a specifically crafted malicious packet, your computer could be commandeered to run a program under the control of the attacking server.

In practice, this attack is going to be limited for most of us for several reasons:

  • By default the Apple implementation doesn't use cryptographically authenticated time packets (flaws one, two and three).

  • If you are connecting to a well-maintained, trusted time server (time.apple.com would be an obvious one), the chances of it going bad or packets from it being intercepted and subverted are low to very low in most circumstances. (The risk of interception and subversion is higher if you don't trust the nameserver or the network, such as unencrypted Wi-Fi in a hostile environment, where you could be directed to a malicious server or someone could try to inject packets as fast as possible to arrive before an authoritative response.)

  • If you're on a Power Mac, unless the attacker knows this and sends a PowerPC-specific sequence, the circulating attacks (to flaws four, five and six) will very probably be designed for x86 and this lowers your effective risk to near zero. At worst ntpd will just crash. (If you're on an Intel Mac, though, this doesn't help you; you're the type of computer an automated scanner would target.)

  • If you don't even try to synchronize your clock, which is apparently the case for more systems than I would have thought, you aren't vulnerable at all. But this isn't a good idea.

If you are running ntpdate from cron only, and not ntpd, ntpdate is potentially vulnerable to flaw six but none of the others (again, assuming authentication is not enabled). No one is even sure if it's exploitable in its current form, and all of the mitigations above (except the last) still apply, so your risk is even lower in that case. I am in error. ntpdate is not vulnerable to this flaw.

In my situation, I have an internal NTP server which talks to a selection of publicly available stratum-1 and stratum-2 timesources. Since it's facing out on the wild wild Internet, I went ahead and upgraded that to ntp 4.2.8, which has these flaws repaired. All of my internal systems only talk to this internal NTP server which is now protected from the problem, and the only way someone's going to subvert that is by either splicing the wires or installing malicious software on the inside -- and if that happens I've got bigger problems than bad clocks. Just in case, however, I went ahead and built and updated ntpdate on my 10.4 systems so that there's no chance.

After careful consideration, I am hesitant to indiscriminately advise everyone to update ntp on your system -- I've learned from offering a rebuild of bash just how many of you can wreck your computers from Terminal :P -- unless you must connect to a risky network from time to time. Even in that case, however, it might be better just to disable "Set date & time automatically" when you're on those types of connections instead and resynch your clock when you're back on something reasonably trustworthy.

If you really can't avoid that, or you're incredibly paranoid, then here is a build of the relevant utilities for 10.4 PowerPC (it will work on 10.5 also). I had to make a change to ntp_io.c to make it build with Xcode 2.5, which I included if you want to roll your own. It only includes the core ntp tools that have relevant vulnerabilities or dependencies (ntpd ntpdate ntpdc ntpq) and none of the SNTP stuff, which is a pretty sucky way to maintain time anyhow. You will notice there is no x86 or universal build, because the current version of the source code doesn't make it easy to build fat binaries, nor instructions, because I want you to think very carefully about why you're installing this. If you're on an x86 system with a vulnerable timekeeper and no available update for this issue (10.6, say), and you're recurrently or even constantly on a hostile network, then you've got bigger problems than this one -- this is a very small bandaid on a potentially deep wound. For that matter, though, even if you're on a Power Mac, please consider what you're trying to accomplish before you install these replacements. They can replace the current ones on the system directly, or if you're running ntpdate from cron and don't have ntpd running, then just put the new ntpdate somewhere convenient like /usr/local/bin and change your crontab to run it from there.

Stay careful.

Wednesday, December 10, 2014

And now for something completely different: My lamps in the Palm of Mom's hand

I am a huge fan of the classic Palm OS. Yes, webOS was pretty cool, while it lasted (there's a Pre 2 on my shelf courtesy of the funkatronic Ed Finkler that I barely got to explore before HP killed the Palm line), but the original Palm OS was not unlike what you'd get if you miniaturized the classic Mac OS into a handheld form factor. Under the hood, you had very similar system designs and implementations, and Macs were well supported as development tools. Heck, they were even 68K systems, like the original Macs, and the switch to the ARM-based Palm devices was eerily similar to the Power Macintosh transition. The ARM Palm OS even had its own 68K emulator -- in fact, the normal state of the system was to be running 68K code!

Myself I started with a Palm m505 which I bought in medical school (new shortly after its introduction, a substantial expense to a starving student) for calculations and drug databases, at which it excelled. I also rapidly learned how to program it, using a remarkable port of the Lua programming language called Plua, and when I upgraded to a ARM-based Zire 72 everything easily transferred over to the new unit which is a testament to how well the sync and OS were designed. Both of these units still work, by the way, even though the m505's battery is shot and won't hold a charge anymore. Even after I got an original iPhone in 2007 I still used the Zire 72 for a number of years because it had better pharmacy apps and it could record video (the iPhone couldn't do this until the 3GS).

Parenthetically, however, if I were forced to pick the best Palm OS device ever made I would actually make a non-Palm selection: the AlphaSmart Dana wireless. If you'd ever wondered what a PalmOS laptop would wind up like, that's pretty much what the Dana is, with a wide 560x160 backlit greyscale LCD, a full and luxurious keyboard, two, count 'em, two SD/SDIO slots, up to 16MB of memory, built-in WEP 802.11b, an integrated USB cradle and even a USB port for a printer. It runs on rechargeable batteries (easily replaceable) or even AAs; mine has a nearly new rechargeable battery pack, a 1GB SD card (maximum supported) in one slot and a Palm Bluetooth SDIO card in the other. The keyboard is incredibly good. There are writers who cling to these things even now because they run for ages on a single charge, the built-in editor is quick and distraction-free, and when you've got your stuff typed, you simply plug it into your Mac or PC and "squirt" it into your word processor of choice: it emulates a USB keyboard! To this day my mother uses one for her notes at church because it runs longer than many laptops, it's light and durable, and when she comes home she can merely plug it into her Mac mini and dump everything into Microsoft Word. Its chief flaw -- and this is a big one -- is that the 33MHz Dragonball VZ CPU is an absolute slug. While it's more than adequate for its avowed purpose and very thrifty with power, heavy duty apps just drag on it even if you install an overclocker.

The biggest, baddest Palm of them all is of course the T|X ("TX"), with a 312MHz Intel PXA270 ARM CPU, 32MB of RAM, 128MB of flash memory (remember that earlier Palms kept everything in RAM, so battery failure was catastrophic), built-in Bluetooth and WiFi and a beautiful 320x480 (320x448 effective resolution) colour LCD screen. Although the earlier Tungsten T5 had 256MB of Flash and a 416MHz CPU, it was cursed by God with system bugs that updates did not completely fix, bad memory management, sluggish Bluetooth and no Wi-Fi. The TX can easily be overclocked to 520MHz (don't exceed this, though, or you'll have a nice doorstop for a day or two while the battery discharges!), it can take additional SD card space (there's even a third-party SDHC driver), and it has more advanced networking, an updated OS and an updated version of the Blazer web browser. By the way, hats off to Dmitry Grinberg for making both of those tools free. That's classy.

But I still like my Zire 72 (in the 72s form factor with the more durable silver paint job) better than my T|X for four reasons: first, the camera, which is pretty dire by today's standards but is still very handy; second, the extra side button (more on that in a moment); third, mini-USB is much easier to work with than that stupid Athena connector on the T5 and TX (though you can charge over USB with the Athena); and last but not least, the replaceable battery. The T|X battery is soldered in -- if you need to hard-power-cycle it, you'll either have to wait it out or cut the leads, and you'll need to get out your soldering iron to put in a new one. The Zire's battery can be (carefully) removed and replaced, however, greatly expanding its longevity. While it won't surf the web anywhere near as well as the T|X, no one's realistically using their Palm for that anymore, you can put nice big SD cards in it as well (and overclock it too because the Zire 72 uses exactly the same CPU as the TX, and the same 32MB of RAM, though no flash memory), and it has built-in Bluetooth also. If you really need WiFi, the palmOne 802.11b SDIO card will work, and the 320x320 screen is pretty decent. Right now, my Zire 72, Dana wireless and TX are all out in the commons area with their own charging stations and they each have their own little tasks they specialize in.

Which brings me to the point of this blog post. I am also a big fan of the Philips hue light system, which uses controllable colour LED bulbs for automated illumination. You can run it from a smartphone, but I centrally control mine on the secured internal network using huepl, a command-line tool I wrote for this purpose. (It works on 10.4, of course.) I have two hue base stations set up, one in my office, and one shown here out in the commons.

Last time Mom was over, Dad and I were working outside early while she slept in my guest room. She got up and found out she couldn't turn the lights on because she didn't have a device to talk to the central server. Never leave your mother in the dark. Ever. So for their next visit I decided I need to make a guest light controller.

While Philips makes a wall switch (interestingly using the kinetic energy of pressing its buttons to send a signal to the hue base station), a custom wireless solution would be more optimal and more flexible. However, I don't allow Wi-Fi on my secured internal network because I don't want someone sneaking into it one day -- it's a secured network for a reason to protect all these lovely old machines, not to mention my financial records. There's a shorter range way of getting into a network though:

Yep, that's a Bluetooth access point. Although theoretically another Class 1 device could try to talk to it from outside the house (if they figured out the pairing code), in practice it's going to be limited by most transceivers being substantially lower power. But more importantly, this specific access point, the Belkin F8T030, is particularly useful in this situation because with its default firmware it only offers the older Bluetooth LAN Access Profile, not the Personal Area Network (Bluetooth PAN) profile currently supported by 10.4+ and most current mobile devices. My Android phones and Macintoshes can only see the Belkin if they're right on top of it, and even then, even with the correct pairing code, they can't do anything with it or connect to the internal network through it. Only the Palm OS devices can -- because they speak Bluetooth LAP, not PAN.

You can flash the F8T030 to speak PAN as well, but I actually don't want that, in this application. By the way, the F8T030 runs uCLinux -- you can even telnet into it and get a shell prompt!

So that solves the connectivity end; now I have to get the Palm to talk to the lights. In this case I selected the Zire 72 as the light controller because I have a couple of them as spares lying around and as mentioned I can repair them to some extent. First thing was to configure it and pair it with the Bluetooth access point, which was pretty simple in PalmOS 5. However, rather than teaching the Palm how to speak the hue base station REST API (because I'd have to give it keys and keep those current), I decided to create a couple scripts on the internal interface of the web server and use EudoraWeb to control the lights through those scripts, like so:

This worked, but you can rapidly see some disadvantages. First, there's not a lot of room left to add more options without getting cluttered (admittedly my choice of the Zire 72 made this problem more acute). Second, running it through EudoraWeb means I don't have any real control over the interface. If the Bluetooth connection got wacky (say Mom didn't have the access point in a direct line of sight, or she pointed the Zire 72 in another direction), EudoraWeb would start throwing generic error messages or offering to switch to offline mode, which would likely drive her crazy and leave the Palm in an "indeterminate" state. And she'd still be in the dark.

So that brings us back to Plua, because I could at least put better error handling in. Plua has very simple built-in primitives for things like streams and TCP/IP sockets, so I moved the scripts to the internal gopher server to make the protocol simple and threw together a proof of concept. This first draft had some big buttons you could push like a remote. This seemed like a good idea and worked well ... except that if the Bluetooth connection failed or you turned the Palm off while the light app was running, Plua went haywire and didn't reconnect. (Obviously a bug, but I don't have any way of fixing that in the Plua runtime.) If you restarted the app, it worked and reestablished the Bluetooth link, but that was inconvenient.

Then it dawned on me: since the connection would be reestablished when the app restarted, move the selection interface into the Palm launcher with multiple separate apps for each light profile that just called the appropriate script on the internal gopher server and quit. Plus, I could have as many profiles as fit on the screen and generate them off a template. On the iMac G4 I wrote a script generator and cross-compiled four basic profiles (off, all on, all dim, corner only) in MacPlua and pushed them over to the Zire 72.

Now, whenever you start any of them (large enough to be pressed with a finger), the Bluetooth link is reestablished if it lapsed, and the correct script on the gopher server is invoked:

But wait: we can make it even easier for Mom. Remember that single side button for voice memos that the Zire 72 has? You can change it to anything, so I changed it to ... turn the lights on by calling that light profile app. Mom picks up the Palm, sees the big label on it saying "to turn lights on, press side button," sees the only side button the device has, presses it, the Zire turns on, the app starts, connects, and the lights come on. You can even find it in a dark room because of the charging light on the end table. See it in the photograph above?

It's like the best Mother's Day present I've ever gotten her. Totally. No doubt.

Here's the source code for the template so you can see how cool and easy Plua is. PalmOS in the house yo.

host = "gopher-internal"
port = 70
sel1 = "/auto/lights/on"

gui.title("Sending command to lights...")

eh, et, es = io.open("tcp:/"..host..":"..port, "rw")
if (eh ~= nil) then
    eh:write(sel1.."\r\n")
    -- Consume all data
    while true do
        ev = gui.event()
        if ev == ioPending then
            buf = eh:read(8192)
            break
        end
        if ev == appStop then
            break
        end
    end
    eh:close()
    -- Fall through to launcher
else
    gui.alert("Can't access network: point Palm at desk")
    -- Fall through to launcher
end

Speaking of the classic Mac OS, I discovered this surprising project last week. I'll just leave it here for your interest.

Monday, December 8, 2014

The POODLE bites (the POODLE chews it) (plus: Fix those bugs, Frenchie)

Let's begin our discussion with a dramatic sound. Wait, it doesn't play? Did you find the MP3 option yet?

During the continuing Bataan death march rewrite of IonMonkey (done with the assembler and about 60% done with the macroassembler) I discovered a glitch in PPCBC with certain comparison operations that the JIT tests apparently don't cover. I don't believe this has any material effect, but we'll fix it anyway, along with a bug in hyphenation that appears to be another Mozilla goof where they assume every architecture represents boolean variables as bytes (on 32-bit PowerPC the native type is 32-bit word, though the compiler can be configured otherwise). That one may affect Linux too, so we'll push it upstream.

Also, during the 29 timeframe I had planned to reduce browser.sessionstore.max_{windows,tabs}_undo as part of a more aggressive tuning of browser memory usage; I don't know why this never got into the code for 31ESR, but it didn't even though the rest of the garbage collection changes did. My original plan was to set the maximum tab and window undo level to 2 each, but a few of you thought two-tab undo was a bit too low, so the current plan is 4 tab undo and 2 window undo (the default is 10 and 3). This means the browser only keeps four old tabs and two old windows in memory, so old objects get released more often. You can, of course, change the old settings back in about:config, but advise if even these default seem unreasonable to you in the comments.

In other news, the two-month-old POODLE SSL/TLS attack just gets worse and worse. Yeah, you heard me: the POODLE exploit can now affect TLSv1 as well, not just SSLv3. POODLE is a padding oracle attack that takes advantage of a gap in the SSLv3 specification where an attacker can arbitrarily manipulate the unvalidated padding bytes in an SSL record. By causing a user's browser (through JavaScript, typically) to issue multiple carefully crafted SSL requests using a credential or login cookie, a malicious intermediary can slowly (with probability 1/256 for each request revealing one byte of a secret to be decrypted) duplicate that credential or cookie and steal it. Today's attack demonstates the same attack is feasible against as many as 10% of the Internet's servers, even those that implement TLS v1.2 (the current spec) as a minimum, due to a similar glitch where the server software doesn't validate those same padding bytes even though they're supposed to. Affected sites even include some major banks and government agencies!

This sort of activity is likely to push up the timetable for deprecating TLS prior to 1.2 or with non-AEAD ciphers due to the increasing evidence that earlier TLS versions and ciphersuites have multiple potentially exploitable cryptographic weaknesses. Firefox (and of course TenFourFox) has supported TLSv1.2 with a full built-in AEAD ciphersuite since 27, and as of this release (31.3) SSLv3 is disabled as well, so TenFourFox is fully current with modern best encryption practise. The same cannot be said for other browsers on PowerPC OS X which depend on the built-in operating system SSL support (this includes Safari and iCab even if Leopard WebKit is installed); the built-in SSL/TLS support libraries in 10.4 and 10.5 are both vulnerable to BEAST and POODLE-type attacks and do not implement TLSv1.1 or TLSv1.2. In the near future browsers and other applications depending on these libraries will no longer be considered safe for secure data transmission, meaning apps will need to package their own encryption libraries to stay current (which we already do, using Mozilla NSS). That's also bad news for Classilla: although attacks like POODLE are highly impractical against Classilla users, that's mostly a factor of the slower JavaScript and network code and not any strength in its crypto suite. While the improved TLSv1.0 support in 9.3.3 has been mostly glitch-free (one site remains under investigation), TLSv1.1 and TLSv1.2 will require some substantial refactoring, and Classilla has no AEAD ciphers at all.

Friday, November 28, 2014

Looking for music to test with?

A couple people asked about some "known working" sites to play with. So here's one: Matt Gray's Kickstarter for Reformation, his remix album of his greatest SID compositions for the Commodore 64 (of which the inferior AY chips can only dream, he says elbowing Martin). The Soundcloud snippets work on that page, as long as you don't seek within the track. Try them. They play great on my 1GHz iMac G4. If you love chiptune remixes like I do, back the project! I'm level 3, and if we hit the next stretch goal, everybody gets another disc!

Also, Amazon Music (FKA Amazon Cloud Player) works with MP3 TenFourFox, but you still can't seek within the track (my bug), there's a quick snatch of music when switching tracks because of a race condition in the HTML5 audio stream (Amazon's bug), certain tracks are not properly detected as MP3 (Mozilla's bug), and if you skip to the next track if the current one is still streaming, it has to stream all the way before the next track will play (also Mozilla's bug). I used it to listen to Pink Floyd's new and allegedly final album The Endless River, which doesn't have much posthumous Richard Wright content after all and is overall weak and meandering even considering the general gestalt of the band's post-Waters incarnation and David Gilmour's avowed goal of an ambient album. Basically, it's a slower and even less adventurous On An Island, and it's kind of a letdown after the big pre-release buildup.

If these don't work for you, something's wrong, but it won't be fixed until the 38 betas. Sorry.

Thursday, November 27, 2014

31.3.0 available

31.3.0 is available (downloads, release notes). Happy Thanksgiving.

This version includes the partial work on MP3 support I completed before halting it due to various streaming bugs in Firefox 31. It is not enabled by default, and because of all the people who didn't read that plugins were unsupported, I'm not including the pref name here (and I'll delete it out of the comments if I see it, so don't even bother). The support is good enough to play Amazon Music if you don't jump around tracks, Soundcloud, and a number of other services. However, seeking within an MP3 file only works for playing from the beginning (if you seek within a track otherwise, it will cause an error and you may need to reload the page), and sites like Amazon may have delays switching tracks due to a bug in how 31ESR manages network streams. These bugs will not be fixed in 31. Do not file reports. If you tell people about this support but fail to tell them that it's unsupported and they open trouble tickets, not only will the tickets be closed on sight, but I will find you and kill you. Please don't make my life more difficult. Enjoy it to the extent that it works, and expect a better implementation in Fx38.

The MP3 support, however, does not make MPEG-4 support any easier really; that probably will require some interfacing to QuickTime or GStreamer and therefore substantially more work. In the short term, I'm altering the browser to pretend that it supports MPEG-4 so that sites will try to present it as an option for HTML5 video which the QTE can then play. This "fake MP4" support is also in this version and works well enough to fool YouTube -- to the point where YouTube doesn't even offer the WebM stream! -- but doesn't yet fool Vimeo. It's also disabled by default. Don't bother reporting bugs in it either until 38.

Now, eat turkey and watch football unless you're not in the United States, in which case you should download the browser and get back to work. It will become official Monday PM Pacific.

Monday, November 24, 2014

Electrolysis means splitsville

Mozilla continues their work towards multi-process Firefox ("Electrolysis" or e10s), and recently enabled it by default for nightly builds. It is still incredibly buggy and Mozilla is not porting the change forward to aurora or beta builds ... yet. However, it is probably the highest priority project at Mozilla and e10s as the default for release builds is going to happen sooner or later; I'm projecting early after Firefox 38, possibly as early as Firefox 39. Once this occurs, single-process Firefox will likely persist as a backup for one or two versions at most. When single-process no longer takes place even in safe mode, that's the end of the transition.

10.4 will not build Electrolysis without writing support for issue 66, and based on some code I've been reviewing, possibly not even then. There are lots of workarounds for threading problems in 10.6 which will almost certainly plague 10.4 and 10.5, quite probably other problems with the older operating systems, and then potential endian issues with serializing data over IPC. If Mozilla had gotten e10s working for Firefox 4 as originally planned, there was a good chance we could have made it work then since Mozilla still supported 10.5 and most of the code was still appropriately endian-agnostic, but they didn't, and we won't. 10.6 support is likely to end in the very near future as well, which dooms us further.

As a result, I'm declaring now that in its current state, Electrolysis will not be supported by TenFourFox. I don't think it can be done on our older OSes in a satisfactory fashion, let alone by one guy, so if and when Mozilla makes it mandatory that's where source parity will end. Assuming no breathtaking technical breakthrough on my part, if it appears to be imminent after the 38ESR diversion (which is my current estimate), we go to feature parity at the end of 38ESR.

Remember, feature parity does not mean the browser stops developing -- it just stops mirroring an official branch of Firefox. By 38 we will have the strongest suite of HTML5 and ECMA6 support available for OS X on PowerPC, including <picture> support and WOFF2 webfonts, and I'm still working on getting IonMonkey PowerPC complete as well as our MP3 playback module. Should I cut support there, I also intend to continue backporting relevant security and stability fixes to the extent they apply, as well as necessary updates to our encryption suite so that HTTP/2 and TLSv1.3 are fully supported. Suitably maintained, we should still have a top-tier layout engine for at least one to two years after the end of ESR38, and a functional one for some time after that. Watch this space.

The release of 31.3.0 got delayed to December 2, which is annoying, because I spent the weekend building it instead of watching the calendar. Fortunately, no patches relevant to us so far have landed afterward, so these already-built versions will be uploaded later this week assuming there are no last minute changes. Happy Thanksgiving.

Thursday, November 20, 2014

Google's not the default ... yahoo?

Mozilla announced today that they are no longer using Google as the search engine default, changing the agreement that's been in place since 2004, and switching to Bing Yahoo!. I've observed on this blog the conflict of interest that Mozilla has, receiving nearly 85% of their income from what is now their largest market competitor (Google, of course, offering Chrome), so this move really only makes sense. Google will still be available as an option, and you can make it your default like it was before, just not the default default. Erm, by default. In return, Mozilla gets the payola and a guarantee that Yahoo! will honour the Do-Not-Track header over their five year deal; we don't know how much payola, but Google paid Mozilla somewhere north of a $100 million a year, so we must assume it's in a similar ballpark to keep them solvent.

(Speaking of Bing, I've switched from Google Maps to Bing Maps. It's much faster because it's prerendered like Google Maps used to be. Also, in the not-so-coincidental category, Google's new Inbox service doesn't work on Firefox.)

This change is going down in December, probably for Fx35, but it's not clear if this will land on the 31ESR branch. That said, I have no good reason not to honour the change whether it lands on 31 or (from our perspective) 38; I don't get paid by either one of them and they don't care whether we use one or the other, so I've got no dog in this fight. If you don't like this, set your default appropriately. As always, if you want to donate, the best way is to click on the ads on this very blog and let Google pay me on your behalf.

Wednesday, November 19, 2014

I deplore my Chromeadore 64

As expected, Chrome 39 dawned and ended 32-bit support on OS X. While it still supports 64-bit on 10.6, expect Snow Leopard support to end entirely as just a matter of time. Blink-cum-Chromium still works on 32 bit systems, because Windows still has a 32-bit Chrome, but I can't imagine Google will keep it around much longer either.

Some of you have inquired about why there is no TenFourFox 31.2.1, and the reason is the issue it fixed (incomplete pave-over installation) doesn't apply to us. I'm waiting for build tags to land and then we'll pull and build 31.3 for the weekend. I've gotten most of the new assembler for IonMonkey PPC completed, and once I get 36 aurora to work, then I will turn my attention to the macroassembler.

Finally, from the world of the Web, a primer on booting Leopard from a USB stick. This probably works for Tiger too. Note the interesting limitation he discovered that 10.7+'s Disk Utility cannot create Apple Partition Maps anymore, meaning they can no longer create bootable volumes for Power Macs. Keep your Snow Leopard Intel Macs humming. :( Seems to work on Mavericks and (reported) Yosemite, so maybe it's just Lion?

Sunday, November 9, 2014

Happy birthday to us

Celebrating the fourth anniversary of our first release, 4.0b7, on November 8, 2010. Here's hoping for another!

Thursday, November 6, 2014

35 up

Typing this in TenFourFox 35. There's a couple of intermittent glitchy things with web fonts I haven't figured out and can't reproduce reliably, but the browser builds and as you can see, basically works. The next step will be to try an opt build once I've finished banging the debug build around to make sure our libvpx hacks stick, and then if I have time start on the Ion-MIPS port to carry this along to Aurora 36.

Sunday, November 2, 2014

Let's get a big bowl of hashes

The hashes document is now up at Google Code, as previously promised. In addition, the changesets for 34 are available from SourceForge, and I just finished landing them on 35 this weekend. We'll see if that build gets off the ground and depending on the time left whether to attack 36 early or start in on the Ion-MIPS conversion.

Tuesday, October 28, 2014

And now for something completely different: Classilla is back

(If you're reading this in Classilla, turn on slow scroll mode or hold down Command as you scroll. I'm still trying to figure out why the browser won't repaint on this blog by itself.)

It is no secret that Classilla has gotten backburnered because of TenFourFox, and to my regret, but there's only one of me. While we have had some substantial code contributors to this project, especially Tobias and Ben but many others, Classilla has only had a couple small patches from outside contributors and none to the core code. That is undoubtedly because it's much harder to get a build running and much harder to work with when you do, but that doesn't change the fact that it's almost entirely been a single nerd effort and this particular nerd has had bigger fish to fry. TenFourFox is the browser I use on my personal systems more than 90% of the time (Classilla and regular Firefox on my token Intel systems takes up the remainder), and I have a rule that I work hardest on the software I actually use myself.

Still, I love the classic Mac OS for its simplicity and its ease of use. Its technical underpinnings and limited multitasking may occasionally be frustrating, and everyone knows that if you push it too hard you'll be forcibly rebooting for your troubles. But there are to this day advantages to the lovely Platinum UI that OS X has not replicated and given its new direction probably never will, including its better human interface consistency, window shading, and true spatial Finder. Lots of applications and great games won't run on anything else, and so Classilla still meets a need for me today on those systems I've dedicated to running OS 9. Plus, I chafed a little bit at that Ars article, since Andrew didn't even try to explore its admittedly unusual features. ;)

So here is Classilla 9.3.3, a long delayed update. It's not the browser I wanted to release; what I wanted to release was a regular security rollup, including fixing a long-standing problem in JavaScript principals, and the partial work on that which did not build required a non-trivial rollback to get it back to compiling. Once that was done, the biggest amount of backlogged complaints received through Report-A-Bug (yes, I do actually read these) was about HTTPS sites that came up with a mysterious error -8182. A quick dive in the source code shows that the closest error base was -8192 (in security/nss/lib/util/secerr.h) and that SEC_ERROR_BAD_SIGNATURE was +10 ... which equals -8182. When the certificates for the offending sites were examined, they were all signed with SHA-256, which is the new certificate standard due to potential weaknesses in current SHA-1 certificates. In fact, by 2017, all three major browsers will no longer regard SHA-1 certificates as trusted and no major website will be using them, so this was critical support to add or at least mitigate.

My first thought was to allow this fatal error to become a non-fatal error that someone could override, similar to forcing acceptance of certificates that were expired or the wrong domain name. In fact, I did implement this (using a different error code so that SHA-1 certs that didn't validate would still fail with a fatal error code), because one day there will be a certificate signing algorithm beyond even the whole of SHA-2. But by 2017, virtually every certificate would fail this test because there would be almost no SHA-1 (let alone MD5) certificates left, meaning certificate verification would essentially not take place for any site. As a future-proofing stopgap, that's one thing, but there must be a better solution in the meantime. How much work would it be to add this support so that Classilla's NSS could actually do the verification?

Interestingly, even though the NSS (Network Security Services) library in Classilla is old, roughly equivalent to 3.7.8 with patches, there was a SHA-2 implementation in its hash algorithm suite that had never been hooked up to anything. In fact, Classilla didn't even have it in its CodeWarrior project files. It compiled and appeared to support everything necessary for a complete SHA-2 implementation (including SHA-256, SHA-384 and SHA-512 variants), so I sat down and noted everywhere that SHA-1 was referenced and duplicated that using the same API. And what the heck -- it worked. Plus, it's a template if SHA-384 or SHA-512 certificates become more common; they can be implemented in exactly the same way. At some point SHA-1 certificate support will also be sunsetted in Classilla, and will be treated as an unknown algorithm, but I will probably not do this until well after the major browsers no longer accept them either.

Server Name Indication was a tougher one. SNI is an extension to the initial TLS handshake that allows virtual hosts on the same server to have different certificates; Mozilla did not support it until Firefox 2.0 (Gecko 1.8.1). Without this support, site hostnames will not appear to match, requiring a fallback to that override dialogue box I'm trying to avoid. Digging through old issues I found a patch in Bugzilla that seemed to fit the bill in a simple sense, but not only did it not work, it made the browser incredibly unstable. NSS can be very byzantine to understand with earlier versions festooned with magic constants and offsets, and one wrong move will upset the entire apple cart. So I pulled that out and tried to hack in some support of my own, but as soon as I turned it on, HTTPS hosts complained my handshake was bad. I had to use openssl on my POWER6 as a dummy server to debug what was going back and forth over the wire until I figured out where my byte offsets were going wrong after a marathon 12 hour hack session. Once this was done, suddenly a great many sites started working even better, to the point where we can ourselves deprecate the flawed SSLv3 in favour of TLSv1 as well (and SSLv2 is now, finally, disabled by default).

I'm not sure how much more I can cram into Classilla NSS. It supports Diffie-Hellman exchange for forward secrecy, but it has absolutely no support for elliptic curve variants (for example, it supports DHE-RSA-AES256-SHA, but not ECDHE), which are more complex but much more performant for a given bit length; implementing this means not only porting over the hashing algorithms but also writing up the TLS hello extension used to negotiate them. Session ticket support is possible, but requires some substantial refactoring under the hood, and there is little payoff given the effort required. TLSv1.1 similarly is possible, but will require a lot of reworking, and many features of TLSv1.2 such as Galois/Counter Mode would have to be implemented from scratch (of terribly questionable utility on a platform where most systems are uniprocessor). However, TLSv1.3, currently in draft, will remove support for non-AEAD ciphers and Classilla's cipher suite is all non-AEAD, so being just short of a complete rewrite it would be nearly impossible to implement it in any reasonable time period. At some point the combination of MD5 and SHA-1 used in TLSv1 will be considered similarly weak, and TLSv1 will also be deprecated, though I don't expect that for some time yet to come. For the next few years even sites that eventually implement TLSv1.3 will still downgrade to TLSv1 and Classilla will still be able to access them. In fact, it is the only classic Mac OS browser now that can.

Compared to those two big updates, the remaining changes are relatively pedestrian. There are some new Byblos stelae for Bing and DuckDuckGo, and Byblos stelae can be stored in Documents:Byblos: so that you can keep the custom ones you write between updates. I fixed issues with custom user agents, wallpapered an annoying layout crash and refreshed user agent strings and SSL certificates. Plus, mail and news is a little faster by not constantly loading an ancient 404 URL from Mozilla (I'm sure their network ops office appreciates the load of thousands of classic Macs being lifted).

For 9.3.4 I intend to return to the security rollup I wanted to complete, and hopefully the hiatus between versions will be much less this time. Layout is still going to be limited, but as Byblos improves I intend to rely more on it to help sites help themselves. If you have a Byblos stele that works well for you, please submit it. I also really want to figure out what the heck is up with this blog layout and scrolling!

Classilla was tested on 9.2.2 on the big 1.8GHz MDD G4, 9.2.2 on my strawberry iMac with a 600MHz Sonnet HARMONi G3, 9.2.2 on my Twentieth Anniversary Macintosh with a 500MHz G3, 9.1 on my 7300+G4/800, 9.1 on my 1400+G3/466 and 8.6 under Rhapsody Blue Box on the Wally G3. Download it now.

It was nice to get the browser and a wonderful classic OS working on the modern web once again, and even though TenFourFox is my priority, I am delighted to say that Classilla is still my joy.

Friday, October 24, 2014

Do not purchase if tamper-evident seal is damaged

Leviathan Security this week managed to catch, red-handed as it were (the malicious server appears to be in Russia and is even named, puckishly, kulak), a Tor exit node actively patching software and modifying it as it came over the wire. Long hypothesized but now finally demonstrated in the wild, this particular node was looking for uncompressed Portable Executable (Windows) binaries and monkeypatching them on their way back to the requester. In fact, it was so active that it gave itself away by trying to patch nearly anything that looked like a Windows application; a smarter node would only patch specific binaries and pass others unmodified, meaning these stealthier malefactors may never be discovered by security researchers even if they do in fact exist.

This applies to unencrypted downloads, of course; assuming no flaws allowing a man-in-the-middle attack, this sort of screwing around would be immediately obvious on a TLS connection. On Google Code, we had encrypted downloads, but Google Code (meanie) no longer allows uploads (scum); SourceForge's mirror network only supports regular HTTP downloads, which are not encrypted (please upvote this ticket if you have an SF account).

Are we high risk? No. You probably have a higher risk of developing Ebola than getting a tainted TenFourFox. It's hard to patch a compressed binary in flight, so it would almost certainly be a complete substitution, it would have to be done on a machine that can build such binaries and it would need to do something interesting on a Power Mac, and given all those preconditions frankly I'm not aware of someone who hates me that much except possibly the cute shop girl at the AT&T store who won't give me her phone number.

Still, I'd like to guard against that possibility, or of any network-imported code over an insecure channel (network snippets, for example, can be loaded into the start page -- which has chrome privileges -- and this should make you suddenly feel very icky and vulnerable). This ties in nicely with a recurring thought I've had of redoing the new tab and start pages so that we're not getting snippets from Mozilla anymore about browser things that don't really apply to us or aren't otherwise compatible. To that end, I think publishing a verifiable hash for all downloads and redoing the new tab (possibly just using a blank one?) and start page for 38ESR will achieve that. We also need to ensure the following:

  • It should not require any special software to verify the hash. You should be able to do it with the tools that come with 10.4.11.
  • The hash list needs to be secure, or that could be tampered with too.
  • The new tab and start pages should not do network loads.

Here's how we'll accomplish that:

  • The most secure hash algorithm that comes with Tiger's built-in OpenSSL appears to be RIPEMD-160; it doesn't support SHA-2, or I'd use that. RIPEMD-160 is a less common hash function but to date cryptanalysis has not demonstrated collisions so far with known attack methods (here's some light bathroom reading on that topic). You can verify that the .zip file downloaded properly by simply going into Terminal and typing (this is the actual hash):

    % openssl dgst -ripemd160 TenFourFox7450-31.2.0.app.zip
    RIPEMD160(TenFourFox7450-31.2.0.app.zip)= e6637dff473f68d2b4a4b6b920c70183d8658dea

  • While we can't upload to Google Code any more, we can certainly post things. The Google Code project page is secured by SSL -- https://tenfourfox.googlecode.com -- and the hashes will either be listed there or on one of the wiki pages. You can grab the hashes securely before you update, download the file from SourceForge, and check that the hash matches. If it does, you have excellent assurance it has not been tampered with in transit. Hashes will be posted for the changesets too.

  • We'll be taking the network and dynamic code out, which as a side benefit will reduce their overhead, and possibly some other minimizing changes. I'll probably still allow the old pages by an about:config switch, though I haven't really plotted how the infrastructure will look either. Just expect this for the next ESR.

Assuming no one finds any glaring holes in this idea, I'll probably implement the hashes for 31.2.0 and the 34 changesets this weekend in my copious spare time. It's just a little bit of extra insurance for our users who may be in more difficult circumstances.

Thursday, October 23, 2014

34 up

I'm typing this in 34 beta, finally, after a large merge where Mozilla destroyed almost all of the old Nitro macroassembler framework that was used for JaegerMonkey and YARR way back when. I restored the pieces we still use, hacked them to be independent and modified PPCBC to point to those. That works as well as 33 did, at least, and it still passes tests. The remainder of the build was relatively uninteresting except for some moved files and a crash introduced with printing that I backed out (it's a Windows-specific change in cross-platform code we don't actually need for our old printing API), though I haven't tried to make an opt build.

Mozilla has two big, though probably survivable, changes for 35 and 36. For 35, Mozilla is updating libvpx, a change that historically tended to upset our custom AltiVec VP8 decoding and will undoubtedly need tweaking to reassemble the fragments. For 36, though, Mozilla is merging DOM and content together, a major seismic shift to unify two core portions of Gecko that have been separate components in the source code even going back to the antediluvian Netscape days. Ironically, this second larger change is the least likely to be problematic for us because it's almost all cross-platform code which we don't have many patches in, but it may invalidate some sections of code indirectly.

Either way, I think I'd like to start the process early on these in order to have extra time for the expected additional merge work, so 34 is going to be changesets-only too; I'll do a little more fiddling with it and then upload them this weekend. I may release 35 as an aurora since that will necessarily require opt builds to be done, though I want to start on the port of MIPS-IonMonkey as soon as I'm done with 35 or 36.

Tuesday, October 21, 2014

Well, it was a good idea at the time

The MP3 idea is shelved for the moment mostly due to bugs in how 31ESR manages media streams (these bugs are also in the regular Firefox, so they're a Mozilla problem). While the MP3 decoder is mostly written and working except for arbitrary seeking which is still problematic and has been disabled for the moment, there are various glitches in streaming music services that make it not ready for 31 (especially on Amazon Music, which is what it was originally written for). These nuisances don't make it unplayable, but they do make it annoying, such as streams failing to time out and requiring full buffering, audio glitches with seeking when tracks switch, and some tracks that fail to play at all. Regular ESR 31.2.0 shows all these problems too, so it's not us, but I can't find a specific fix for these and the proper reworking was probably spread along multiple fixes around Fx33 or Fx34.

Because it mostly works but fails in subtle ways, I will leave it in 31.3 but disabled. To avoid a situation with plugins (this is why we can't have nice things) where people post instructions but don't link back to posts explaining limitations in support, you'll have to find the setting yourself. Seeking doesn't work (except for replaying from the beginning) and there are other bugs which I won't fix in the 31 timeframe, so don't even bother reporting them. No further work will be done on it until we get to 38, assuming we do. If you give these instructions out and you don't remind people of this, I will cut out your intestines with a rusty spoon.

The 34 port hopefully will be done by this weekend and then I need to figure out what to do with it.

Tuesday, October 14, 2014

Poodles are fine dogs. SSLv3 is not a fine protocol.

For the hat trick, the security researchers who brought you BEAST and CRIME now introduce POODLE, the "Padding Oracle On Downgraded Legacy Encryption" attack. The attack allows an SSLv3 connection to be gradually broken down using a dramatically faster method that coughs up a byte of the session cookie on average one out of 256 tries. Ironically, because the demonstrated attack vector is JavaScript, our JIT makes us more vulnerable to this because we run JavaScript faster.

There is no fixing SSLv3 against this -- it's a fundamental flaw in the protocol that can't be worked around, and while previous weaknesses were mitigated somewhat with cipher changes there's no easy way out this time. Most browser vendors, including Mozilla, are taking this as the last straw and ending support for SSLv3, especially since TLS v1 and its successors have been around so long. Even Classilla supports TLS v1.

What will happen is that starting in Firefox 34, all connections must use TLS v1 or higher, giving servers a six-week cycle to do any necessary upgrades. We will follow suit: if Mozilla does not do this first, we will also set TenFourFox on the corresponding ESR release (31.3) to do the same. If you want to do this early, go into about:config and set security.tls.version.min to 1; if this causes problems with HTTPS sites you visit, switch back and tell them to get with it. Classilla users are advised to disable SSLv2 and SSLv3 in the Preferences window, under Security (leave TLS checked). The next version of Classilla, if and when I get those SSL changes done, will have them defaulted off as well.

34 is about 80% done (slogging through JavaScript), and the MP3-enhanced 31 is now playing substantially more audio files without problems. There is still a critical issue with parsing MP3 metadata and it's still quite crashy, but it's developing pleasingly quickly.

Monday, October 13, 2014

31.2.0 available, and music to your ears

31.2.0 is now official.

Also, now I can make that announcement: tonight, TenFourFox played its first MP3 audio file properly with its own internal decoder in-browser (using a modified version of minimp3, a heavily distilled-down version of ffmpeg's MP3 decoder component). Although Firefox can play MP3s using OS X AudioToolbox, the AudioToolbox in 10.4 does not support MP3, so we had to roll our own implementation. Although the codec is still under patent, it's old and we're non-profit and the code has been part of Linux distros for a long time, so we might be able to sneak by without anyone caring. (I hope I won't regret this.)

This code is not ready for primetime: it can't seek within a stream yet, it asserts all over the place with some MP3 files -- I suspect another lurking endian problem in Mozilla's built-in MP3 frame parser since I've already had to fix one of them to even get this far -- and it crashes if you try to queue up two files, though this is a memory allocator issue with its decoder tables and I have an idea how to fix it. It's also kind of slow, though this is partially because I'm on a debug build, partially because the code is not very optimized, and partially because Mozilla disables bigger buffering with audio-only streams, all of which are solveable.

However, the payoff is now music streaming sites like Amazon Cloud Player will "just work." (Amazon Music complains it needs Flash, but it really doesn't. Disable Flash on an Intel Mac running Firefox and you'll see.) And actually that's what I wrote it for, because now OmniWeb crashes trying to run Amazon Music, which is what I used to use on the G5 in the background. I write code to scratch my itches and the priorities for TenFourFox are decided accordingly. If there is a particular feature you want, I strongly urge you to sit down and write it and submit it. Ahem.

Please note that this doesn't really improve MP4 support; the codec glue is quite different and will have to be hacked in separately. I don't have a timeframe for MPEG-4 support and it is not currently a priority for me because the QTE meets my use cases currently. You, of course, can contribute. Ahem, I say.

The target is 31.4.0, though it will ship disabled in that version, so you'll have to turn it on. If it works well, we'll loosen it for 31.5. However, I still need to get 34 working first, and I still want to complete our IonMonkey implementation. I'm determined to get to 38 before concluding parity.

Friday, October 10, 2014

31.2.0 available

31.2.0 is now available (downloads, release notes). It has no changes other than security updates and stability fixes. However, I made a breakthrough on a long requested feature and it may be possible to introduce this into 31.3 or 31.4 -- we'll see, more later. As usual, it will become live Monday night.

Meanwhile, the patches are down for 34 and I'm about 60% through getting a debug build up, but this weekend I'm taking some time off to enter a pinball tournament. Don't wait up.

Thursday, October 9, 2014

And the bash is probably done: 4.3.30

I'm not quite sure if the glitches fixed are exploitable with all the other armour, but 4.3.30 is now available, and should conclude the Shellshock chapter at least this time around. The previous post has been updated, since everyone's still linking to it.

Tuesday, October 7, 2014

And now for something completely different: bad blocks, bad trip

You've already met thule, my recently resurrected Macintosh IIci running NetBSD. The peril of running something old is that things fail. After I got a new recapped motherboard in it, now the hard disk started throwing a couple bad sectors.

Bad sectors happen; magnetic media occasionally gets glitchy. (I do have a backup, but I like running things until they die, so I'm going to keep this drive running until it don't run no more.) Modern drives can often recruit from a pool of sectors and transparently redirect a write to a bad sector to a good sector, but no hard drive can do this on read if the media is bad, and on a drive this geriatric (a 2GB Quantum Fireball from oh-my-gawd-it's-old) even the former is not a given. We want some way to tell the operating system to never use those sectors again.

Some vintage operating systems do this more or less overtly. For example, my Alpha Micro Eagle 300 has an actual bad block file that tells the operating system hands off these sectors (one wonders what would happen if a bad block occurred in the sectors that actually contain the bad block file). Classic Mac OS doesn't really support this, though tools like Norton Disk Doctor can quietly allocate them out of reach. However, Norton Disk Doctor doesn't understand a NetBSD FFS volume. NetBSD does have a tool called badsect(8) that can take a list of sectors and create bogus file descriptors that fsck_ffs will turn into invalid files soaking up those sectors; similar tools exist on other BSDs and even some Linux distributions. This is a little scary and there's not much documentation on this because there aren't many freaks like me running NetBSD on something this old. So here goes.

The console will tell you the bad sector (either on screen or through dmesg). Here's the actual output thule was spitting out:

sd0(ncrscsi0:0:0):
Check Condition on CDB: 0x28 00 00 32 3d 61 00 00 01 00
    SENSE KEY:  Media Error
   INFO FIELD:  3292513
     ASC/ASCQ:  Unrecovered Read Error

sd0(ncrscsi0:0:0):
Check Condition on CDB: 0x28 00 00 32 3d 62 00 00 01 00
    SENSE KEY:  Media Error
   INFO FIELD:  3292514
     ASC/ASCQ:  Unrecovered Read Error

sd0(ncrscsi0:0:0):
Check Condition on CDB: 0x28 00 00 32 3d 63 00 00 01 00
    SENSE KEY:  Media Error
   INFO FIELD:  3292515
     ASC/ASCQ:  Unrecovered Read Error
Eeewww.

What this means is that relative sectors 3292513-5 inclusive are kaput. I noticed this because the periodic full system backup had started failing on certain files, so I knew which files were bad, and I could restore them from the previous backup. But before I could do that, I wanted to make sure the OS didn't try to reuse the bogus sectors.

First, I rebooted it in single user mode. This is very important; you are tinkering with a live filesystem and run not only the risk of kernel panic but serious data loss if other things are writing to the disk while you are. (Yes, I've panicked systems at least once by cheating on this. Learn from my mistakes.) Next, I ran fsck on the disk and allowed it to remove the wrecked files, generating more media errors in the process. It did so, and instructed me to reboot. But we don't want to do that yet; we still have those sectors to account for.

Now we use badsect. We create a directory /BAD and tell badsect to create the placeholder files for those crashed sectors:

badsect /BAD 3292513 3292514 3292515

BE VERY CAREFUL. There is no going back. If you type the sector wrong, you may only waste a perfectly good unallocated sector, or you may destroy a file mid-stream. Check your typing twice, and then check it again. If you did this right, the sectors will be marked and badsect will tell you do run fsck again. Do so.

This time, fsck will notice the bogus files and ask HOLD BAD BLOCK? The answer is yes. It should ask you each time for each sector. If you correctly reserved all the bad sectors, there should be no more media errors on the console. If the bad sectors were part of the file system, you may have some DUPs to resolve, which you should answer yes as well if it asks.

Now the part that threw me initially: when fsck proceeds to the second phase and checks pathnames, it will (correctly) notice that the files in /BAD point to bad blocks ... and offer to delete them! Do not let it!

DUP/BAD I=277769 OWNER=root MODE=100600
SIZE=1024 MTIME=Oct 7 19:58 2014
FILE=/BAD/3292513

REMOVE?

The answer this time, and for anything in /BAD, is NO -- or you'll remove the linkage to the bad sector, and you'll have to create it all over again. This will be true when you fsck this volume again, and you will, because bad sectors will eventually occur somewhere else.

By the end, after you've allowed fsck to complete all the remaining salvage, you shouldn't see any more media errors on the console. Now reboot and restore the files that were lost, if they were important, and tell your backup script to exclude anything in /BAD.

I'm toying with resurrecting thule a second time, but this time running A/UX. That might be fun.

Thursday, October 2, 2014

Blog roll

Nothing new to report on TenFourFox right now -- still working on the patches for 34 -- but two blog posts from the retro-Mac community I wanted to highlight:

Riccardo Mori's rebuttal on doing practical work on classic Macs. If you're reading this blog, you're probably already sympathetic to his view, and since Riccardo is very complimentary to TenFourFox and Classilla my linking to his post is totally, unapologetically and shamelessly self-serving, but I think he hits the main points very well (original article for comparison).

Martin Kukač prepares his G5 for another decade, which is his retrofit of an older G5 useful to those of you trying to maintain your air-cooled DP systems. Fortunately, my quad is already ready to go another nine innings.

Wednesday, October 1, 2014

And the bash goes on again: 4.3.28

bash 4.3.28 is now available, fixing CVE-2014-7186 and CVE-2014-7187, and this should repair all the known outstanding problems. Since everyone is linking to the original post, I have updated it with the new self-tests and instructions.

Monday, September 29, 2014

Sunday, September 28, 2014

Ars goes back to the future

Ars Technica subjected Andrew Cunningham to getting his work done on an 800MHz TiBook (in OS 9 and in 10.5). Admittedly, I, too, would find an 800MHz TiBook a bit trying, though I would not have installed Leopard on it. My slowest daily driver is the 1GHz iMac G4 and even that doesn't do much more than basic browsing. My 12" iBook G4/1.33 is about the slowest I'd like to be using for typical work. Fortunately, the quad still acquits itself very well!

Thursday, September 25, 2014

Bashing bash one more time: updated universal 4.3.30 covering all known bash flaws

FINAL UPDATE: 4.3.30 is now available. There are no new tests, and it is not clear the flaws it fixes are exploitable with the other changes, but it is available for those that wish it. Assuming no other vulnerabilities are found in the near future, this should be the last patch.

UPDATE THE THIRD: To clarify some questions, 4.3.28 is not vulnerable to CVE-2014-6277 and -6278, as well as all the other reported vulnerabilities. See Michal Zalewski's post on the subject.

UPDATE THE SECOND: It's a bug bash! Updated to 4.3.28 below with more fixes for two more internal vulnerabilities (CVE-2014-7186 and -7187). Hopefully this should be the last one for awhile. Just overlay it over your old copy of 4.3.2x.

UPDATE: bash just keeps on giving. Updated to 4.3.27 below with more steps to repair this vulnerability. Just overlay it over your old copy of 4.3.26.

See the previous entry, but in short, bash has been shown to have a pretty nasty little vulnerability that causes it to inadvertently execute shell commands in the environment you pass it. This attack does work on Power Macs because most shell commands are cross-platform, and appears to exist on all versions of OS X.

The solution is easy: build a new bash from the newly patched source code. As a service to you, I have done so, and compiled it for PowerPC and Intel so it will also work for users on 10.6 who are not receiving updates either. See above The version earlier today had a preliminary version of the patch which does not fix a second variant vulnerability. This version does. If you used one of the "build from source" tricks that were circulating earlier today (MacRumors, etc.), your version does NOT have this second issue patched. Either wait for the public source trees to update and rebuild it (likely early tomorrow), or use this one.

The bash these steps will install works on 10.4 all the way to 10.9 on 32-bit Intel, 64-bit Intel and PowerPC. It requires no other dependencies. The idea is to replace your system bash -- yes, you can use Homebrew, Tigerbrew, MacPorts, etc., to get an updated copy, but your built-in bash is still vulnerable unless you replace it. This is designed to accomplish that. WARNING AGAIN: If you are not comfortable with the Terminal, get someone to help you!

  1. In a Terminal.app window, verify that you have a vulnerable system so that you can see what that looks like (the command is all one line):

    env x='() { :;}; echo vulnerable' bash -c "echo this is a test"

    It should print

    vulnerable
    this is a test

  2. Check the second vulnerability. This creates a file called echo with the date in it, if your system is vulnerable:

    env X='() { (a)=>\' sh -c "echo date"; cat echo

    It should print something like (the messages and of course the time will vary):

    bash: X: line 1: syntax error near unexpected token `='
    bash: X: line 1: `'
    bash: error importing function definition for `X'
    Thu Sep 25 22:12:49 PDT 2014

    (Delete the file it makes before you continue! rm echo)

  3. Check the third vulnerability.

    env foo='() { echo not patched; }' bash -c foo

    It should print

    not patched

  4. NEW: Check the fourth vulnerability.

    bash -c 'true <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF' || echo "vulnerable"

    It should print (the exact number of lines may vary):

    bash: line 1: warning: here-document at line 1 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 1 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 1 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 1 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 1 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 1 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 1 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 1 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 1 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 1 delimited by end-of-file (wanted `EOF')
    Bus error
    vulnerable

  5. NEW: Check the fifth vulnerability.

    bash -c '( for x in {1..200} ; do echo "for x$x in ; do :" ; done ; for x in {1..200} ; do echo done ; done )' | bash || echo "vulnerable"

    It should print (the exact number may vary):

    bash: line 129: syntax error near `x129'
    bash: line 129: `for x129 in ; do :'
    vulnerable

  6. This has now been patched to 4.3.30. This has now been patched to 4.3.28. This has now been patched to 4.3.27. Download the patched bash 4.3.26. Put it in your home directory. If necessary, double-click to decompress it so that you have a file in your home directory called bash-4.3.30-10.4u.

  7. Close all terminal windows and programs just to make sure you won't stomp on bash while a program is trying to call it. Start Terminal and have exactly one window open.

  8. In that terminal window:

    • exec tcsh
    • chmod +x bash-4.3.30-10.4u

      If you replaced /bin/bash (and/or /bin/sh) with any earlier version of these patched bashes, DO NOT DO THE NEXT TWO COMMANDS. If you have never replaced them, go ahead; these will put the old ones in a safe place, just in case.

    • sudo mv /bin/bash /bin/bash_old (enter your password)
    • sudo mv /bin/sh /bin/sh_old (enter your password if needed)

      Everybody does these:

    • sudo cp bash-4.3.30-10.4u /bin/bash (enter your password if needed)
    • sudo cp bash-4.3.30-10.4u /bin/sh (enter your password if needed)

  9. Test it stuck by trying the statements again:

    env x='() { :;}; echo vulnerable' bash -c "echo this is a test"

    It should print

    this is a test

    Now, try the second one:

    env X='() { (a)=>\' sh -c "echo date"; cat echo

    It should print

    date
    cat: echo: No such file or directory

    If this test appears to fail, make sure that you delete the file echo (rm echo) first, and test it again.

    Now, try the third one:

    env foo='() { echo not patched; }' bash -c foo

    It should print

    bash: foo: command not found

    Now, try the fourth one:

    bash -c 'true <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF <<EOF' || echo "vulnerable"

    It should print (there should be exactly 14 lines, one for each <<EOF):

    bash: line 1: warning: here-document at line 0 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 0 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 0 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 0 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 0 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 0 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 0 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 0 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 0 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 0 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 0 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 0 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 0 delimited by end-of-file (wanted `EOF')
    bash: line 1: warning: here-document at line 0 delimited by end-of-file (wanted `EOF')

    Now, try the fifth one:

    bash -c '( for x in {1..200} ; do echo "for x$x in ; do :" ; done ; for x in {1..200} ; do echo done ; done )' | bash || echo "vulnerable"

    Nothing will appear. However, if you want a positive test, do the following:

    bash -c '( for x in {1..200} ; do echo "for x$x in ; do :" ; done ; for x in {1..200} ; do echo done ; done )' | bash && echo "not vulnerable"

    It should print:

    not vulnerable

  10. Restart your Mac as a paranoia to make sure everything is using the new copy of bash.

  11. Bask in the glow. Then, find a shell that doesn't suck.

Bashing bash: updated universal OS X bash available as a public service

UPDATE THE SECOND: People are unclear about what I mean by "incomplete." I mean, the existing patch for bash is not complete. Even for the list of commands floating around to build from source (seen on MacRumors and other sites, which requires Xcode), you won't be protected from the second variant of the attack. You'll have to wait for that final patch. Your call whether to continue with the below. Updated to fix /bin/sh as well.

UPDATE: Looks like the fix is incomplete; someone found a way around it. I'll post an update with the newer version when they decide on it, but you can fix the immediate bug now with the steps below.

I hate bash. I prefer tcsh. But Apple made it the default, and now bash has been shown to have a pretty nasty little vulnerability that causes it to inadvertently execute shell commands in the environment you pass it. This attack does work on Power Macs because most shell commands are cross-platform, and there will be no update for us. The vulnerability appears to exist on all versions of OS X.

I thought about this for awhile on how much of an attack surface we're exposing. Servers that might run shell scripts as CGIs would be very high risk, but if you're actually using a 10.4 or 10.5 machine as an externally facing web server you really need your head examined. Similarly, there's the possibility of getting privileged setuid scripts on a multiuser system owned, which again would be primarily an issue for servers.

The risk is less clear on single user workstations, the situation in which I imagine most Power Macs exist, which either run bash purely as a login shell or certain programs may call out to it to accomplish certain system tasks. Realistically, I don't see a large attack surface here, but clever little sneaks might find a way and besides, the solution is easy: build a new bash from the newly patched source code. As a service to you, I have done so, and compiled it for PowerPC and Intel so it will also work for users on 10.6 who are not receiving updates either.

The bash these steps will install works on 10.4 all the way to 10.9 on 32-bit Intel, 64-bit Intel and PowerPC. It requires no other dependencies. The idea is to replace your system bash -- yes, you can use Homebrew, Tigerbrew, MacPorts, etc., to get an updated bash, but your built-in bash is still vulnerable unless you replace it. This is designed to accomplish that.

  • WARNING: The following steps assume you know how to use Terminal.app and some basic Unix commands. If you don't, get help.

    1. In a Terminal.app window, verify that you have a vulnerable system so that you can see what that looks like (the command is all one line):

      env x='() { :;}; echo vulnerable' bash -c "echo this is a test"

      It should print

      vulnerable
      this is a test

    2. This has been superseded Download the patched bash 4.3.25. Put it in your home directory. If necessary, double-click to decompress it so that you have a file in your home directory called bash-4.3.25-10.4u.

    3. Close all terminal windows and programs just to make sure you won't stomp on bash while a program is trying to call it. Start Terminal and have exactly one window open.

    4. In that terminal window:

      • exec tcsh
      • chmod +x bash-4.3.25-10.4u
      • sudo mv /bin/bash /bin/bash_old (enter your password)
      • sudo cp bash-4.3.25-10.4u /bin/bash (enter your password if needed)
      • sudo mv /bin/sh /bin/sh_old (enter your password if needed)
      • sudo cp bash-4.3.25-10.4u /bin/sh (enter your password if needed)

    5. Test it stuck by trying the statement again:

      env x='() { :;}; echo vulnerable' bash -c "echo this is a test"

      It should print

      bash: warning: x: ignoring function definition attempt
      bash: error importing function definition for `x'
      this is a test

    6. Restart your Mac as a paranoia to make sure everything is using the new copy of bash.

    7. Bask in the glow. Then, find a shell that doesn't suck.

  • Wednesday, September 24, 2014

    What the security issue was all about

    Mozilla has released 32.0.3 and the official advisory, so now we can talk about it. What got fixed in 31.1.1 is a flaw in verifying signatures of RSA certificates, most importantly such as those used to confirm the identity of secure sites. By exploiting this bug the proof of concept caused Firefox to accept a forged certificate, which facilitates information stealing and man-in-the-middle attacks. The forgery is an interesting variant of a well-understood older attack vector called the Bleichenbacher attack, named after the crytographer who discovered it, or more colourfully the "million message attack," itself a specific form of an adaptive chosen-ciphertext attack. Through a combination of flaws, a clever attacker could synthesize a completely bogus "valid" certificate in a relatively small amount of computing time and use it to impersonate victim servers to steal credentials and data.

    The general type of flaw suggests other crypto libraries may be vulnerable to this specific problem or a related form of it. That said, this problem has existed in Mozilla code since at least 2006; earlier versions don't use the same ASN.1 parsing code, but almost certainly have other problems related to certificate verification. You are strongly advised to update, because the relative ease with which a certificate can be forged will put yourself at much greater risk in the near future if you don't (24.7.0 is vulnerable, as is every prior version of TenFourFox).

    31.1.1 available

    Downloads of 31.1.1 are available -- release notes are pending Firefox's release since the security information is embargoed. Please try them on your system. They will become live as soon as Mozilla does so for mainline Firefox and ESR. I'll have more information for you then.

    Tuesday, September 23, 2014

    Heads up: 31.1.1 imminent

    Mozilla is chemspilling for a high-priority security issue which we are vulnerable to; this will force a 31.1.1 build, on which I will also take all the current security and stability fixes Mozilla has landed on 31ESR to date. The G5 is currently chugging out release candidates hopefully for release to you the testers tomorrow evening to become live on Thursday.

    Saturday, September 20, 2014

    For Chrome, 32 bits are not enough

    Google is touting its 64-bit build of Chrome for OS X, scheduled for rollout with Chrome 39 in November. Almost a whisper by comparison, though, is what will happen to the 32-bit build. Well, it's very simple: it won't exist. This is the end for the 32-bit Intel Mac as far as Google is concerned, even though those Macs can run 10.6, which Chrome, at least for the moment, still supports. Chromium might still build, but I wouldn't bet on it.

    Besides the obvious -- no 32-bit browser -- there are some other knock-on effects, the most important being that 32-bit plugins won't work either. This is not a big deal presently since the major plugins have 64-bit versions now, but Google is completely removing NPAPI plugin support later this year as well. This leaves its perverse little PPAPI plugin support, which only Chrome can run, to maintain Flash compatibility using its bespoke in-browser version; Mozilla, Microsoft and Apple have all uniformly refused to support PPAPI. That's okay with Google, because soon their Native Client implementation is going to take over your desktop with Android apps anyway.

    As it stands, Firefox is a fat x86 binary with 32-bit and 64-bit versions, but historically when Google has withdrawn support Mozilla has been quick to follow, and Google's decision is already being discussed on the Mozilla developer channels. Currently the only thing really forcing 32-bit support is the need for Silverlight for Mac, which doesn't exist in a 64-bit version. Since Chrome won't be supporting NPAPI plugins of any kind soon, they don't have that limitation; should Mozilla find a HTML5 workaround for sites like Netflix, or the userbase drop enough, you can confidently expect the end of 32-bit support on OS X from Firefox as well.

    This is important for us in two respects: we rely on a lot of 10.6 code still, and a 64-bit only build clears the way to end 10.6 support; and we are a 32-bit hybrid Carbon/Cocoa application. The only 64-bit Power Mac is the G5, but even if I limited TenFourFox to the G5 systems exclusively (which I won't do, because at a minimum I myself use the 7450 build also), it still wouldn't work because there is no 64-bit Carbon and 10.4's 64-bit Cocoa support is incomplete. As long as 32-bit builds persist, 10.6 support is likely to remain as well, because they serve no purpose on 10.7+. Once they disappear, I predict Mozilla will drop both 10.6 and 10.7 simultaneously and support only 10.8+. (Remember, Firefox already can no longer be built on 10.6, even though it will still run on it.) We could probably still make much of the cross-platform code work on 32-bit PowerPC because the Linux and Windows tier-1 builds still support 32-bit, at least for the time being, but the OS X-specific portions would become much harder to work with.

    On that note, due to my dissatisfaction I've decided to scotch the 33 release and jump to working on 34. (If you want the changesets to build your own to play with, you can get them from the unstable folder on SourceForge. Please note that they are against 33-aurora, which is now 33-beta, so you should expect some merging to be required.) For now, I'm just concentrating on getting to 38, and secondarily on completing the MIPS JIT conversion to PowerPC; 34 is currently crippled by a severe bug on systems without hardware acceleration, but this is biting Mozilla as well, so it is virtually certain they will fix it shortly. 38 is kind of a nice milestone because by the end of 38ESR support, even the last quad G5 to roll off the assembly line will have received a full 10 years of support, from 2006 to 2016. I'd really like to make that, at least, and pick up the important HTML5 and ECMA6 features that are being added such as <picture>.

    In other news, I picked up an Apple Workgroup Server 9150 and restored it this weekend. I'm sort of developing a hyperspecialized interest in Apple server-grade hardware, such as it existed, particularly the truly unique Apple Network Servers; while I don't really have much interest in the Workgroup Servers in general because they were rebadged workstation Macs with occasionally different storage loadouts, the WGS 9150 is a legitimately odd chimera of the Quadra 950/Apple Workgroup Server 95 and the Power Macintosh 8100 with some strange attributes of its own. In its prototype form as Green Giant, it was also the testbed for Apple's brief and sorrowful flirtation with NetWare (the so-called "Wormhole" project); it went over so badly with testers that Apple ended up developing Shiner in its place, which became the ANS, and the Green Giant prototype was recycled into the 9150 with MacOS. After stripping it down, cleaning its profoundly grimy contents, applying a bit of steel wool and gentle care to get the yellowing and deep marks out of the plastic, and then replacing the CD-ROM, adding another hard disk and recabling it, it is now sitting next to the G5 quietly installing MkLinux. Why MkLinux, you ask? Because it's period-appropriate, it probably doesn't suck on the 601, Apple actually supported it, and I want to compare it to Rhapsody. Watch for an "ANFSCD" on it in the near future.

    Finally, I am concerned that SeaMonkey PPC has halted updates; I haven't seen anything since May of this year. Our Tenfourbird builder from the Land of the Rising Sun has been keeping pace, but SeaMonkey PPC is based on its own code, and it's possible he hasn't been able to keep it working. Hopefully that's not the case, since the looming end of the original WebKit1 will also limit Leopard WebKit users, though I know Tobias will continue to work on fixes and improvements. We need more and better browser support for OS X/ppc, not less.