Friday, June 29, 2018

Ad-blocker-blockers hit a new low. What's the solution?

It may be the wrong day to slam the local newspapers, but this was what greeted me trying to click through to a linked newspaper article this morning on Firefox Android. The link I was sent was from the Riverside Press-Enterprise, but this appears to be throughout the entire network of the P-E's owners, the Southern California News Group (which includes the Orange County Register, San Bernardino Sun and Los Angeles Daily News):

That's obnoxious. SCNG is particularly notorious for not being very selective about ads and they tend to be colossally heavy and sometimes invasive; there's no way on this periodically green earth that I'm turning the adblocker off. I click "no thanks." The popover disappears, but what it was covering was this:

That's not me greeking the article so you can't see what article I was reading. The ad-blocker-blocker did it so that a clever user or add-on can't just set the ad-blocker-blocker's popover to display:none or something. The article is now incomprehensible text.

My first reaction is that any possibility I had of actually paying $1 for the 4 week subscription to any SCNG paper just went up in the flames of my great furious wrath (after all, this is a blog s**tpost). The funny part is that TenFourFox's basic adblock actually isn't defeated by this, probably because we're selective about what actually gets blocked and so the ad-blocker-blocker thinks ads are getting through. But our old systems are precisely those that need adblockers because of all the JavaScript (particularly) that modern ad systems lard their impressions up with. Anyway, to read the article I actually ended up looking at it on the G5. There was no way I was going to pay them for engaging in this kind of behaviour.

The second thought I had was, how do you handle this? I'm certainly sympathetic to the view that we need stronger local papers for better local governance, but print ads are a much different beast than the dreck that online ads are. (Yes, this blog has ads. I don't care if you block them or not.) Sure, I could have subscriptions to all the regional papers, or at least the ones that haven't p*ssed me off yet, but then I have to juggle all the memberships and multiple charges and that won't help me read papers not normally in my catchment area. I just want to click and read the news, just like I can anonymously pick up a paper and read it at the bar.

One way to solve this might be to have revenue sharing arrangements between ISPs and papers. It could be a mom-and-pop ISP and the local paper, if any of those or those still exist, or it could be a large ISP and a major national media group. Users on that ISP get free access (as a benefit of membership even), the paper gets a piece. Everyone else can subscribe if they want. This kind of thing already exists on Apple TV devices, after all: if I buy the Spectrum cable plan, I get those channels free on Apple TV over my Spectrum Internet access, or I pay if I don't. Why couldn't newspapers work this way?

Does net neutrality prohibit this?

Friday, June 22, 2018

TenFourFox FPR8 available

TenFourFox Feature Parity Release 8 final is now available (downloads, hashes, release notes). There are no changes from the beta except for outstanding security patches. As usual, it will go live Monday night, assuming no changes.

Saturday, June 16, 2018

TenFourFox FPR8b1 available

TenFourFox Feature Parity Release 8 beta 1 is now available (downloads, release notes, hashes). There is much less in this release than I wanted because of a family member in the hospital and several technical roadblocks. Of note, I've officially abandoned CSS grid again after an extensive testing period due to the fact that we would need substantial work to get a functional implementation, and a partially functional implementation is worse than none at all (in the latter case, we simply gracefully degrade into block-level <div>s). I also was not able to finish the HTML <input> date picker implementation, though I've managed to still get a fair amount completed of it, and I'll keep working on that for FPR9. The good news is, once the date picker is done, the time picker will use nearly exactly the same internal plumbing and can just be patterned off it in the same way. Unlike Firefox's implementation, as I've previously mentioned our version uses native OS X controls instead of XUL, which also makes it faster. That said, it is a ghastly hack on the Cocoa widget side and required some tricky programming on 10.4 which will be the subject of a later blog post.

That's not to say this is strictly a security patch release (though most of the patches for the final Firefox 52, 52.9, are in this beta). The big feature I did want to get in FPR8 did land and seems to work properly, which is same-site cookie support. Same-site cookie support helps to reduce cross-site request forgeries by advising the browser the cookie in question should only be sent if a request originates from the site that set it. If the host that triggered the request is different than the one appearing in the address bar, the request won't include any of the cookies that are tagged as same-site. For example, say you're logged into your bank, and somehow you end up opening another tab with a malicious site that knows how to manipulate your bank's transfer money function by automatically submitting a hidden POST form. Since you're logged into your bank, unless your bank has CSRF mitigations (and it had better!), the malicious site could impersonate you since the browser will faithfully send your login cookie along with the form. The credential never leaked, so the browser technically didn't malfunction, but the malicious site was still able to impersonate you and steal your money. With same-site cookies, there is a list of declared "safe" operations; POST forms and certain other functions are not on that list and are considered "unsafe." Since the unsafe action didn't originate from the site that set the cookie, the cookie isn't transmitted to your bank, authentication fails and the attack is foiled. If the mode is set to "strict" (as opposed to "lax"), even a "safe" action like clicking a link from an outside site won't send the cookie.

Same-site cookie support was implemented for Firefox 60; our implementation is based on it and should support all the same features. When you start FPR8b1, your cookies database will be transparently upgraded to the new database schema. If you are currently logged into a site that supports same-site cookies, or you are using a foxbox that preserves cookie data, you will need to log out and log back in to ensure your login cookie is upgraded (I just deleted all my cookies and started fresh, which is good to give the web trackers a heart attack anyway). Github and Bugzilla already have support, and I expect to see banks and other high-security sites follow suit. To see if a cookie on a site is same-site, make sure the Storage Inspector is enabled in Developer tools, then go to the Storage tab in the Developer tools on the site of interest and look at the Cookies database. The same-site mode (unset, lax or strict) should be shown as the final column.

FPR8 goes live on June 25th.

Saturday, June 9, 2018

Let's kill kittens with native messaging (or, introducing OverbiteNX: if WebExtensions can't do it, we will)

WebExtensions (there is no XUL) took over with a thud seven months ago, which was felt as a great disturbance in the Force by most of us who wrote Firefox add-ons that, you know, actually did stuff. Many promises were made for APIs to allow us to do the stuff we did before. Some of these promises were kept and these APIs have actually been implemented, and credit where credit is due. But there are many that have not (that metabug is not exhaustive). More to the point, there are many for which people have offered to write code and are motivated to write code, but we have no parameters for what would be acceptable, possibly because any spec would end up stuck in a "boil the ocean" problem, possibly because it's low priority, or possibly because someone gave other someones the impression such an API would be acceptable and hasn't actually told them it isn't. The best way to get contribution is to allow people to scratch their own itches, but the urgency to overcome the (largely unintentional) institutional roadblocks has faded now that there is somewhat less outrage, and we are still left with a disordered collection of APIs that extends Firefox relatively little and a very slow road to do otherwise.

Or perhaps we don't have to actually rely on what's in Firefox to scratch our itch, at least in many cases. In a potentially strategically unwise decision, WebExtensions allows native code execution in the form of "native messaging" -- that is, you can write a native component, tell Firefox about it and who can talk to it, and then have that native component do what Firefox don't. At that point, the problem then becomes more one of packaging. If the functionality you require isn't primarily limited by the browser UI, then this might be a way around the La Brea triage tarpit.

Does this sound suspiciously familiar to anyone like some other historical browser-manipulated blobs of native code? Hang on, it's coming back to me. I remember something like this. I remember now. I remember them!


If you've been under a rock until Firefox 52, let me remind you that plugins were globs of native code that gave the browser wonderful additional capabilities such as playing different types of video, Flash and Shockwave games and DRM management, as well as other incredibly useful features such as potential exploitation, instability and sometimes outright crashes. Here in TenFourFox, for a variety of reasons I officially removed support for plugins in version 6 and completely removed the code with TenFourFox 19. Plugins served a historic purpose which are now better met by any number of appropriate browser and HTML5 APIs, and their disadvantages now in general outweigh their advantages.

Mozilla agrees with this, sort of. Starting with 52 many, though not all, plugins won't run. The remaining lucky few are Flash, Widevine and OpenH264. If you type about:plugins into Firefox, you'll still see them unless you're like me and you're running it on a POWER9 Talos II or some other hyper-free OS. There's a little bit of hypocrisy here in the name of utilitarianism, but I think it's pretty clear there would be a pretty high bar to adding a new plugin to this whitelist. Pithily, Mozilla has concluded regardless of any residual utility that plugins kill kittens.

It wasn't just plugins, either. Mozilla really doesn't like native code at all when it can be avoided; for a number of years in the Mozilla developer community I remember a lot of interest in making JavaScript so fast it could be used for all the media decoding and manipulation that native plugins then did. It's certainly gotten a lot faster since then, but now I guess the interest is having WASM do that instead (I'll wait). A certain subset of old-school extensions used to have binary components too, though largely to do funky system-level things that regular XPCOM in JavaScript and/or js-ctypes couldn't handle. This practice was strongly discouraged by Mozilla but many antiviruses implemented browser scanners in that fashion. This may have been a useful means to an end but binary components were a constant source of bugs when Mozilla altered something that the add-on authors didn't expect to change, and thus they kill kittens as well.

Nevertheless, we're back in a situation where we actually need certain APIs to be implemented to make certain types of functionality -- functionality which was perfectly acceptable in the XUL addons era, I might add -- actually still possible. These APIs are not a priority to Mozilla right now, but a certain subset of them can be provided by a (formerly discouraged) native component.

It's time to kill some kittens.

The particular kitten I need to kill is TCP sockets. I need to be able to talk over TCP to Gopher servers directly on a selection of port numbers. This was not easy to implement with XPCOM, but you could implement your own nsIChannel as a first-class citizen that looked to the browser like any other channel in pure JavaScript back in the day, and I did. That was OverbiteFF. I'm not the only one who asked for this feature, and I was willing to write it, but I'm not going to spend a lot of time writing code Mozilla won't accept and that means Mozilla needs to come up with an acceptable spec. This has gradually drowned under the ocean they're trying to boil by coming up with everyone's use cases whether they have anything in common or not and then somehow addressing the security concerns of all these disparate connection models and eventually forging the one ring to bind them all sometime around the natural heatdeath of the observable universe. I'm tired of waiting.

So here is OverbiteNX. OverbiteNX comes in two parts: the actual WebExtensions addon ("OverbiteNX" proper), and Onyx, a native component that the browser add-on asks to connect to a Gopher server and then receives data from it. Onyx is supported on Microsoft Windows, Linux and macOS (I've included binaries for Win32 and macOS 10.12+), and works probably anywhere else you can compile and run Firefox and Onyx as long as it supports Berkeley sockets. Onyx has no dependencies, is written in cross-platform C with a couple Windows-specific bits, and is currently presented in a single source file in an uncomplicated manner to not only serve the purpose of the extension but also to serve as an educational example of how to kill your own particular kitten. All of the pieces needed to hook it up (the Windows NSI and example JSON) are included and you can see how it connects with the front-end and the life cycle of a request in the source code. I discuss the architecture in more detail and how you can play with it. (If you are a current user of OverbiteWX, please remove it first.)

To be sure, I'm not the first to have this idea, and it's hardly an optimal solution. Besides the fact I have to coax someone to install a binary on their system to use the add-on, if I change the communication protocol (which is highly likely given that I will need to add future support for multithreading and possibly session IDs when multiple tabs are in use) then I need to get them to upgrade both the add-on and the native component at the same time. Users on weird platforms, and as a user of a weird platform I certainly empathize, have to do more work to get it to run on their system by compiling and installing it manually. The dat extension I linked to at the beginning of this paragraph gets around the binary code limitation by running the "native" component as JavaScript under Node.js, but that requires you to run Node, which is an extra step that seems unrealistic for many end-users and actually adds more work to people on weird platforms.

On the other hand, this is the only way right now it's going to work at all. I don't think Mozilla is going to move on these pain points unless they get into the situation where half the add-ons on AMO depend on external components they don't manage. So let the kitten killing begin. If you've got an idea and the current APIs don't let you implement it, see if a native component will scratch your itch and increase the pressure on Mountain View. After all, if Mozilla doesn't add useful WebExtensions APIs, there's no alternative but feline mass murder.

Provocative hyperbole aside, bug reports and pull requests appreciated for OverbiteNX. At some point in the near future the add-on piece will be submitted to AMO, which I expect to pass the automatic scanner since it doesn't do any currently proscribed operations, and then it's just a matter of keeping the native component in sync for future releases. Watch the Github project for more. And enjoy Gopherspace. Just don't tell the EU about it.

(Note for the humourless: The cat pictured is my cat. She is purring on her ottoman as this was written. No cat was harmed during the making of this blog post, though she does get an occasional talking-to. The picture was taken when she was penned up in the laundry while I was out and she was not threatened with any projectile weapon.)

Monday, June 4, 2018

Just call it macOS Death Valley and get it over with

What is Apple trying to say with macOS Mojave? That it's dusty, dry, substantially devoid of life in many areas, and prone to severe temperature extremes? Oh, it has dark mode. Ohhh, okay. That totally makes up for everything and all the bugs and all the recurrent lack of technical investment. It's like the anti-shiny.

In other news, besides the 32-bit apocalypse, they just deprecated OpenGL and OpenCL in order to make way for all the Metal apps that people have just been lining up to write. Not that this is any surprise, mind you, given how long Apple's implementation of OpenGL has rotted on the vine. It's a good thing they're talking about allowing iOS apps to run, because there may not be any legacy Mac apps compatible when macOS 10.15 "Zzyzx" rolls around.

Yes, looking forward to that Linux ARM laptop when the MacBook Air "Sierra Forever" wears out. I remember when I was excited about Apple's new stuff. Now it's just wondering what they're going to break next.

Sunday, June 3, 2018

Another weekend on the new computer (or, making the Talos II into the world's biggest Power Mac)

Your eyes do not deceive you -- this is QEMU running Tiger with full virtualization on the Talos II. For proof, look at my QEMU command line in the Terminal window. I've just turned my POWER9 into a G4.

Recall last entry that there was a problem using virtualization to run Power Mac operating systems on the T2 because the necessary KVM module, KVM-PR, doesn't load on bare-metal POWER9 systems (the T2 is PowerNV, so it's bare-metal). That means you'd have to run your Mac operating systems under pure emulation, which eked out something equivalent to a 1GHz G4 in System Profiler but was still a drag. With some minimal tweaks to KVM-PR, I was able to coax Tiger to start up under virtualization, increasing the apparent CPU speed to over 2GHz. Hardly a Quad G5, but that's the fastest Power Mac G4 you'll ever see with the fastest front-side bus on a G4 you'll ever see. Ever. Maximum effort.

The issue on POWER9 is actually a little more complex than I described it (thanks to Paul Mackerras at IBM OzLabs for pointing me in the right direction), so let me give you a little background first. To turn a virtual address into an actual real address, PowerPC and POWER processors prior to POWER9 exclusively used a hash table of page table entries (PTEs or HPTEs, depending on who's writing) to find the correct location in memory. The process in a simplified fashion is thus: given a virtual address, the processor translates it into a key for that block of memory using the segment lookaside buffer (SLB), and then hashes that key and part of the address to narrow it down to two page table entry groups (PTEGs), each containing eight PTEs. The processor then checks those 16 entries for a match. If it's there, it continues, or else it sends a page fault to the operating system to map the memory.

The first problem is that the format of HPTEs changed slightly in POWER9, so this needs to be accommodated if the host CPU does lookups of its own (it does in KVM-HV, but this was already converted for the new POWER9 and thus works already).

The bigger problem, though, is that hash tables can be complex to manage and in the worst case could require a lot of searches to map a page. POWER8 and earlier reduce this cost with the translation lookaside buffer (TLB), used to cache a PTE once it's found. However, the POWER9 has another option called the radix MMU. In this scheme (read the patent if you're bored), the SLB entry for that block of memory now has a radix page table pointer, or RPTP. The RPTP in turn points to a chain of hierarchical translation tables ("radix tree") that through a series of cascading lookups build the real address for that page of RAM. This is fast and flexible, and particularly well-suited to discontinuous tracts of addressing space. However, as an implementational detail, a guest operating system running in user mode (i.e., KVM-PR) on a radix host has limitations on so-called quadrant 3 (memory in the 0xc... range). This isn't a problem for a VM that can execute supervisor instructions (i.e., KVM-HV) because it can just remap as necessary, but KVM-HV can't emulate a G3 or G4 on a POWER9; only KVM-PR can do that.

Fortunately, the POWER9 still can support the HPT and turn the radix MMU off by booting the kernel with disable_radix. That gets around the second problem. As it turns out, the first problem actually isn't a problem for booting OS X on KVM once radix mode is off, assuming you hack the KVM-PR kernel module to handle a couple extra interrupt types and remove the lockout on POWER9. And here we are.(*)

Anyway, you lot will be wanting the Geekbench numbers, won't you? Such a competitive bunch, always demanding to know the score. Let's set two baselines. First, my trusty backup workstation, the 1GHz iMac G4: It's not very fast and it has no L3 cache, which makes it worse, but the arm is great, the form-factor has never been equaled, I love the screen and it fits very well on a desk. That gets a fairly weak 580 Geekbench (Geekbench 2.2 on 10.4, integer 693, floating point 581, memory 500, stream 347). For the second baseline, I'll use my trusty Quad G5, but I left it in Reduced power mode since that's how I normally run it. In Reduced, it gets a decent 1700 Geekbench (1907/2040/1002/1190).

First up, Geekbench with pure emulation (using the TCG JIT):

... aah, forget it. I wasn't going to wait all night for that. How about hacked KVM-PR?

Well, damn, son: 1733 (1849/2343/976/536). That's into the G5's range, at least with math performance, and the G5 did it with four threads while this poor thing only has one (QEMU's Power Mac emulation does not yet support SMP, even with KVM). Again, do remember that the G5 was intentionally being run gimped here: if it were going full blast, it would have blown the World's Baddest Power Mac G4 out of the water. But still, this is a decent showing for the T2 in "Mac mode" given all the other overhead that's going on, and the T2 is doing that while running Firefox with a buttload of tabs and lots of Terminal sessions and I think I was playing a movie or something in the background. I will note for the record that some of the numbers seem a bit suspect; although there may well be a performance delta between image compression and decompression, it shouldn't be this different and it would more likely be in the other direction. Likewise, the poor showing for the standard library memory work might be syscall overhead, which is plausible, but that doesn't explain why a copy is faster than a simple write. Regardless, that's heaps better than the emulated CPU which wouldn't have finished even by the time I went to dinner.

The other nice thing is that KVM-PR-Hacky-McHackface doesn't require any changes to QEMU to work, though the hack is pretty hacky. It is not sufficient to boot Mac OS 9; that causes the kernel module to err out with a failure in memory mapped I/O, which is probably because it actually does need the first problem to be fixed, and similarly I would expect Linux and NetBSD won't be happy either for the same reason (let alone nesting KVM-PR within them, which is allowed and even supported). Also, I/O performance in QEMU regardless of KVM is dismal. Even with my hacked KVM-PR, a raw disk image and rebuilding a stripped down QEMU with -O3 -mcpu=power9, disk and network throughput are quite slow and it's even worse if there are lots of graphics updates occurring simultaneously, such as installing Mac OS X with the on-screen Aqua progress bar. Minimizing such windows helps, but only when you're able to do so, of course. More ominously I'll get occasional soft lockouts in the kernel (though everything keeps running), usually if it's doing heavy disk access, and it acts very strangely with stuff that messes with the hardware such as system updates. For that reason I let Software Update run in emulated mode so that if a bug occurred during the installation, it wouldn't completely hose everything and make the disk image unbootable (which did, in fact, happen the first time I tried to upgrade to 10.4.11). Another unrelated annoyance is that QEMU's emulated video card doesn't offer 16:9 resolutions, which is inconvenient on this 1920x1080 display. I could probably hack that in later.

QEMU also has its own bugs, of course; support for running OS 9/OS X is very much still a work in progress. For example, you'll notice there are no screenshots of the T2 running TenFourFox. That's because it can't. I installed the G3 version and tried running it in QEMU+KVM-PR, and TenFourFox crashed with an illegal instruction fault. So I tried starting it in safe mode on the assumption the JIT was making it unsteady, which seemed to work when it gave me the safe mode window, but then when I tried to start the full browser still crashed with an illegal instruction fault (in a different place). At that point I assumed it was a bug in KVM-PR and tried starting it in pure emulation. This time, TenFourFox crashed the entire emulator (which exited with an illegal instruction fault). I think we can safely conclude that this is a bug in QEMU. I haven't even tried running Classic on it yet; I'm almost afraid to.

Still, this means my T2 is a lot further along at being able to run my Power Mac software. It also means I need to go through and reprogram all my AutoKey remappings to not remap the Command-key combinations when I'm actually in QEMU. That's a pain, but worth it. If enough people are interested in playing with this, I'll go post the diff in a gist on new Microsoft Visual GitHub, but remember it will rock your socks, taint your kernel, (possibly) crash your computer and (definitely) slap yo mama. You'll also need to apply it as a patch to the source code for your current kernel, whatever it is, as I will not post binaries to make you do it your own bad and irresponsible self, but you won't have to wait long as the T2 will build the Linux kernel from scratch and all its relevant modules in about 20 minutes at -j24. Now we're playing with POWER!

What else did I learn this weekend?

  • If you want disable_radix to stick on bootup, just put it in a grub config and Petitboot will pick it up. This is particularly helpful because I still can't figure out what I should be putting in BOOTKERNFW to enable the Radeon card, so Petitboot still comes up with a blank screen unless I pull the VGA disable jumper.

  • amdgpu is still glitchy sometimes. Trying to view very large images in Eye of GNOME actually caused graphics corruption and garbled the mouse pointer. The session quickly became unusable and I had to bail out and restart X11. Viewnior substituted nicely and doesn't have this problem. (I also found it with Hugin when I was trying to find something to view the equirectangular panoramas taken with my Ricoh Theta cameras. Eventually I hacked FreePV into building and that substitutes nicely as well. I gave it an entry in /usr/share/applications so that I could directly view images from the GNOME file manager.)

  • Symlinking xdg-open to open means I can still open most things from the command line with the same OS X command.

  • I had a dickens of a time getting my Android Pixel XL to talk to the T2. GNOME kept throwing MTP errors when it tried to mount it (yes, the phone was in MTP mode). gphoto2 could list the directories in PTP mode, but couldn't actually transfer anything. Even adb shell would disconnect after just a few commands, and adb pull wouldn't even get past enumerating the file list. Eventually I found some apocryphal note that someone fixed their phone by connecting it over USB 2.0 instead of USB 3.0, so I found a USB 2.0 hub, plugged that in, and plugged the Pixel XL into that. Now it works. (The same cable works fine at USB 3.0 speeds on my MacBook Air with the Pixel, so I am assuming this is an issue with the chipset in the T2. But I also think that's Google's bug, not Raptor's or Fedora's.)

  • The freely distributable Linux fonts are improving but still suck, so I transferred my entire font folder over AFP from the G5. The OTF and TTF fonts worked immediately, and Fondu easily converted the DFONT fonts, but I also have a lot of old Mac fonts and even some font suitcases that are pure resource forks. GNOME doesn't know how to transfer those. I could have tried copying them to a FAT volume and extracting the resource forks from the hidden directory, but that seemed icky. After some thought, I went to the Terminal on the G5 and did this instead:

    ls | fgrep -vi '.ttf' | fgrep -vi '.otf' | fgrep -vi '.dfont' | fgrep -vi '.bmap' | perl -ne 'chomp;print"\"$_\"\n"' | xargs -n 1 -I '{}' macbinconv -mac '{}' -mb '/home/spectre/rfont/{}.bin'

    The little snippet of Perl there preserves embedded spaces in the filenames. When I ran it, it turned all the font resources into MacBinary, I transferred the resulting files to the T2, and Fondu converted them as well. Now I have my fonts.

  • Speaking of fonts, I wanted to play with font hinting on an individual font basis and installed Fonts Tweaks for GNOME to do this. I started with my converted Lucida Grande that I use for most of the display theme, and it eliminated the anti-aliasing and made much of the display painfully jaggy. I couldn't for the life of me figure out how to undo it, even after uninstalling Fonts Tweaks, until I found .config/fontconfig/conf.d/ and removed the offending entry.

  • I was able to get my RadioSHARK to play audio, but it required a little setup first. I first had to compile the shark tool for Linux, which fortunately I had preserved as part of radioSH. That worked with little adjustment, but the new tool kept throwing permissions errors and I didn't really want to run it as root. The problem is that it uses libhid to talk to the HID portion of the RadioSHARK and libhid expects to be able to enumerate any USB device it encounters to see if it's really a HID. Eventually I hit on this udev recipe and added myself to a new usb group:

    SUBSYSTEM=="usb", GROUP="usb", MODE="0660"

    Now shark can control it, and I can listen in VLC (using a URL like pulse://alsa_input.usb-Griffin_Technology__Inc._RadioSHARK-00.analog-stereo).

  • Deadpool 2 is pretty funny. Not as good as the first, because there was no way it was going to out-outrageous it, and the gags were a little forced sometimes, but Brolin made a solid Cable and Domino was outstanding. And hey, a Kiwi kid who can act and isn't stereotyped! I even recognized them driving along the Crowsnest Highway (BC 3) near Manning Park before the merge with Trans-Canada 1 in Hope, a beautiful road I have driven many times myself. Plus, the mid-credits scenes made up for all of the movie's holes (he said, carefully avoiding the joke), and be sure to stay put for a musical laugh-out-loud moment at the very end!
Back on the G5 tomorrow for more work on date-time pickers for TenFourFox FPR8. I've abandoned CSS grid again for the time being because my current implementation is actually worse than not trying to render it at all. That's discouraging, but at least it gracefully degrades.

(*) Given that no changes were made to the HPTE format to get KVM-PR to work for OS X, it may not even be necessary to run the kernel with radix mode off. I'll try that this coming weekend at some point.