TenFourFox FPR9b2 available

TenFourFox Feature Parity Release 9 beta 2 is now available (downloads, hashes, release notes). This version tightens up the geometry on the date/time pickers a little, adds some more hosts to basic adblock, fixes a rare but easily wallpapered crash bug and further tunes up hash tables using a small patch from Firefox 63 (!). I am looking at a new JavaScript issue which does not appear to be a regression, but I'd like to fix it anyway since it may affect other sites. However, I'm not sure if this is going to make FPR9 final, which is still scheduled on or about September 4 due to the American Labor Day holiday on the usual Monday.

The WiFi fix in beta 1 was actually to improve HTML5 geolocation accuracy, and Chris T has confirmed that it does, so that's been updated in the release notes. Don't worry, you are always asked before your location is sent to a site.

On the Talos II side, I've written an enhancement to KVMPPC allowing it to actually monkeypatch Mac OS X with an optimized bcopy in the commpage. By avoiding the overhead of emulating dcbz's behaviour on 32-bit PPC, this hack improves the T2's Geekbench score by almost 200 points in Tiger. Combined with another small routine to turn dcba hints into nops so they don't cause instruction faults, this greatly reduces stalls and watchdog tickles when running Mac apps in QEMU. I'll have a formal article on that with source code for the grubby proletariat shortly, plus a big surprise launch of something I've been working on very soon. Watch this space.

TenFourFox FPR9b1 available

Before we begin: if you haven't seen it, check out the newly updated and refurbished TenFourFox FAQ, and consider this sobering thought on the state of web monitoring advertising.

Also, for those of you who may be unaware, long-time Mozillian Gervase Markham passed away, surrounded by his family. He was ever a professional to the end. I didn't know him as well as some, but I'll always remember him, not least of which for his unwavering faith in the face of adversity and leaving us far too soon. Go with God.

TenFourFox Feature Parity Release 9 beta 1 is now available (downloads, hashes, release notes). There are several new features and many bug fixes in this version. FPR9 is also the first TenFourFox release to pull from the new extended support release, Firefox 60ESR, and we have updated the extended validation certificate roots and our certs'n'pins import script to pull from that source instead of the shortly-to-be-decommissioned 52ESR. All relevant security and stability patches on 60ESR so far have also been backported.

On the bug fix side, there is a crash fix for media tracks from Raphael, updates to the ATSUI font blacklist (mostly for certain Japanese fonts) that can now block incompatible fonts through the CSS Font Loading API as well, updated timezone data for the ICU internationalization library, ICU security fixes, fixes for button sensing in events which should get around some weird glitchy things where the mouse buttons get ignored, and a dumb old bug with WiFi scanning. I also added a layout fix for button subelements that keep getting split apart, fixing problems on sites like Twitch and GoDaddy.

There are also a number of performance related changes. First, the idle observer minimal interval has been increased to be a better fit for our old machines, since doing this every second (potentially) robs CPU time and possibly instruction cache space away from user interaction. Do note, however, that while this reduces work being run while you use the browser the work is only delayed and not eliminated, meaning improvements in responsiveness and animation will be at the cost of memory being occasionally held longer and taking longer to release, and possibly longer GC pauses (though less often), so advise if you don't think the tradeoff is worth it (this is adjustable, though it's hardcoded within TenFourFox and is not a pref). This release also has some tuning to Cocoa events handling to slightly reduce scroll wheel latency and reduce some overhead with custom events, and additional code in JavaScript to make it more nimble about cancelling and recompiling since most of our systems are uniprocessor.

The other big change is to array handling. I backported a number of changes that landed in Firefox 55 which increase the performance of certain array operations such as splices and shifts by anywhere from 35 to 80 times. This does not translate into as dramatic a boost as you might think because in small numbers these types of operations were already reasonably swift, nor are they typically executed in large numbers or in tight loops back to back, but for those sites that do, they will run quite a bit quicker.

Of the new features, CSS column properties are now unprefixed and tested, which will help improve site compatibility. Unfortunately one big new feature that I planned for FPR9 won't make this release: deferred idle callbacks. The requestIdleCallback DOM API was enabled in Firefox in Fx55 and allows browsers to schedule work in the idle time between animation frames. Rarely do any of our old systems have such idle time available, and the 50ms or less usually granted (and actually checked for by the W3C test suite) isn't much time to do much work anyway. However, we can also infer from the timeout that is (optionally) passed that the website is able to tolerate the work being deferred up to that long, so deferred idle callbacks deferred the work up to that long. For example, on YouTube this would delay comments and other pieces of the UI so that the CPU could concentrate more on decoding and blitting frames. My initial thought was to try to run the callback when the system was not under load, but it turned out to be simpler and more reliable to just run it at the maximum timeout interval (or after a default interval if one was not specified). Unfortunately, after a lot of idle callbacks had run test builds started suffering random stalls even after the callbacks had completed, indicating something wasn't getting cleaned up, so it is pref'ed off in this version while I try to figure out what I did wrong. You can play with it by turning tenfourfox.dom.requestIdleCallback.enabled to true, but you'll have to restart the browser if you change that setting, so remember that any assessment you make about performance should keep in mind whether the browser is "fresh" or not. This is something I want to get working but it's not a major issue right now because most of the sites that depend on it (including YouTube) have polyfills.

However, the other big new feature I wanted in FPR9 did land: native HTML5 time and date input controls. On my list of future features likely to be used by sites, these scored very high because for those sites that don't need custom clock/calendar UIs this can eliminate them having to load and maintain one of their own just to select intervals. Date and time input controls were not implemented in Firefox until Fx60, and the implementation Firefox uses is a cross-platform one with a fair bit of JavaScript and other support code for vagaries in date handling and localization. I really didn't want to backport all of that because it would be slower and bloatier and especially because we already have all of this special handling built into Mac OS X itself through NSDate. So I took a tip from the Android version and, in a like manner to the colour picker and file picker, which use native Cocoa widgets, implemented date and time pickers using the built-in NSDatePicker widget. This, in turn, handles all the calendar and date manipulations so we don't have to, displays the calendar and time and date using the user's own locale, and does it all at native speed to boot. Genius!

For simplicity I chose to implement this as a modal window rather than the drop-down Firefox 60 uses, which solved a lot of synchronization issues even though it's not as slick. Ordinarily you'd implement a simple modal window in Cocoa with an NSAlert, and if you want to add something to the alert box, you do so by creating an NSView with the widgets you want to splice in and inserting it as an accessory view.

There's just one problem: NSAlert didn't implement setAccessoryView: until 10.5, so you can't do that in Tiger. Or can you?

When looking for undocumented functionality in Cocoa, the best tools are either class-dump (3.1.2 works with Tiger), or my personal favourite GUI tool, MagicHat. Let's see what MagicHat comes up with for NSAlert:

// Framework:      AppKit
// Header:        NSAlert.h
// Documentation: Unknown

@interface NSAlert : NSObject
{
 int             _alertStyle;
 float           _buttonPadding;
 float           _buttonSpacing;
 float           _buttonSpacingMaxX;
 float           _buttonSpacingY;
 NSArray*        _buttons;
 NSSize          _defaultPanelSize;
 id              _delegate;
 SEL             _didDismissSelector;
 SEL             _didEndSelector;
 NSWindow*       _docWindow;
 id              _first;
 id              _helpAnchor;
 id              _helpButton;
 id              _imageView;
 NSTextField*    _informationField;
 id              _messageField;
 float           _messagePadding;
 NSSize          _minButtonSize;
 id              _modalDelegate;
 NSPanel*        _panel;
 id              _second;
 char            _showsHelp;
 id              _third;
 NSImage*        _unbadgedImage;
 char            _useNSLayout;
 [?]             reserved;
 id              reserved1;
 id              reserved2;
}

+ (id) alertWithError: (id) parameter1;
+ (id) alertWithMessageText: (id) parameter1 defaultButton: (id) parameter2 alternateButton: (id) parameter3 otherButton: (id) parameter4 informativeTextWithFormat: (id) parameter5;
+ (void) stopAllTimersForSpeaking;

- (char) _dontWarnAgain;
- (void) _setDontWarnMessage: (id) parameter1;
- (char) _showsDontWarnAgain;
- (id) addButtonWithTitle: (id) parameter1;
- (int) alertStyle;
- (void) beginSheetModalForWindow: (id) parameter1 modalDelegate: (id) parameter2 didEndSelector: (SEL) parameter3 contextInfo: (void*) parameter4;
- (id) buildAlertStyle: (int) parameter1 title: (id) parameter2 formattedMsg: (id) parameter3 first: (id) parameter4 second: (id) parameter5 third: (id) parameter6 oldStyle: (char) parameter7;
- (id) buildAlertStyle: (int) parameter1 title: (id) parameter2 message: (id) parameter3 first: (id) parameter4 second: (id) parameter5 third: (id) parameter6 oldStyle: (char) parameter7 args: (char*) parameter8;
- (id) buttonPressed: (id) parameter1;
- (id) buttons;
- (void) dealloc;
- (id) delegate;
- (void) didEndAlert: (id) parameter1 returnCode: (int) parameter2 contextInfo: (void*) parameter3;
- (void) didEndSheet: (id) parameter1 returnCode: (int) parameter2 contextInfo: (void*) parameter3;
- (void) finalize;
- (id) helpAnchor;
- (id) icon;
- (id) informativeText;
- (id) init;
- (id) messageText;
- (float) messageWidthForMessage: (id) parameter1;
- (void) placeButtons: (int) parameter1 firstWidth: (float) parameter2 secondWidth: (float) parameter3 thirdWidth: (float) parameter4;
- (void) prepare;
- (char) rememberChoice;
- (int) runModal;
- (int) runModalSheetForWindow: (id) parameter1;
- (void) setAlertStyle: (int) parameter1;
- (void) setDelegate: (id) parameter1;
- (void) setHelpAnchor: (id) parameter1;
- (void) setIcon: (id) parameter1;
- (void) setInformativeText: (id) parameter1;
- (id) setMessage: (id) parameter1;
- (void) setMessageText: (id) parameter1;
- (void) setRemberChoiceMessage: (id) parameter1;
- (void) setShowsHelp: (char) parameter1;
- (id) setTitle: (id) parameter1 andMessage: (id) parameter2;
- (char) showsHelp;
- (void) startSpeaking: (id) parameter1;
- (void) startTimerForSpeaking;
- (void) stopTimerForSpeaking;
- (id) window;
- (void) windowDidBecomeKey: (id) parameter1;

@end

Most of these are documented. Unfortunately there is no obvious analogue to setAccessoryView:, so there's no easy way that way. However, there is an interesting selector called buildAlertStyle:. This is not documented, but the name suggests that it gets called at some point to construct the NSAlert, presumably out of a generic NSWindow. Unfortunately because it's undocumented it's not at all clear what those parameters actually do.

I ruminated over this for a couple of days until by dumb luck I ran across an old library called NSAlertCheckbox. This library, though fragmentary and appearing to hail from around the days of Jaguar 10.2, does exactly what the name implies: it adds a checkbox control to an NSAlert. Well, said I, if it can add an NSButton, it can add a NSDatePicker. I cleaned up and expanded the library and wrote a little model (modal?) application to test my theory. It looked like this:

(Here is the source code for this example. You can compile it with Xcode 2.5 and the Mac OS X 10.4 SDK. Note in this and the below examples, because the model is being run from the command line and not within a regular NSApplication, it cannot intercept keystrokes. Just roll with this for the explanations below.)

The way the library works is by hooking those mysterious functions, which do indeed get called during window construction. Happily it turns out we don't need to know anything about what those opaque parameters are; we can modify the window without them (and just pass them along) by iterating through the components within the NSWindow that makes up the NSAlert. Once the code finds the second text field (the explanatory text, here articulately represented by the deep and meaningful words "Blah blah"), it then inserts a subview after that containing the date picker, et voilá.

This is not enough, however. While we can easily select the date and emit it (using NSDateFormatter), we can't type it, so selecting years and arbitrary months can sometimes require a lot of mousing around. Indeed, if you go to System Preferences to set the date and time, you have the choice of either using a stepper to manually enter a date or selecting one off the calendar. We can support that here too ... by adding another NSDatePicker!

(Source code for this example.)

The trick here was to keep them in sync so that you could type a date and have the calendar widget reflect it. By using a delegate on each date picker's cell, we can make modifications in one picker be synchronized to the other.

That suffices for dates. How about times? Straightforward enough: we just change the style to make the twin NSDatePickers into a clock and time stepper instead, and alter our template to NSDateFormatter to emit the time (in hours and minutes, and optionally seconds). Since the HTML spec also requires that we allow sites to specify minimums and maximums, let's implement that here too:

(Source code for this example.)

Because of the time interval constraint we implemented, when you run it you can only pick a time between 7:30am and 5:45pm; the pickers won't let you select anything else. Exactly what we want.

Before you upgrade to the beta, open either the MDN time or date picker pages and click in one of the demonstration input fields. In FPR8, this is just a text box, and you just enter text. (Same for Safari.) In FPR9, however, when you click or select that field the appropriate picker will pop open and the field will be autofilled for you in the proper format. If you press delete in the field, the date or time will be cleared. If you click Cancel, no change will occur. All of the standard constraints are supported, along with defaults (if no default is specified, the current time and/or date is used). Easy. As a safety measure, though, I implemented a basic filter to prevent NSDateFormatter from being exposed directly to web content since who knows what crap is lurking in there; it's only used as a secondary validator once the web-provided date is known to be in a non-pathological format.

Still to be implemented are month (easily done by just returning the month, not the full date), local date and time (needing a quadruple date picker) and week (harder, but possible). With the exception of local date'n'time, however, these are much less likely to be common and I'll defer those to later versions of the browser.

FPR9 will come out officially parallel to Firefox 62 and 60.2 on September 4, but I wanted a nice long beta test period since there is a lot of stuff in this release and to provide a sufficient timeframe for additional betas if required, so you're getting what I've got now. Bang on it and post your impressions in the comments.

NetSpectre: not much of a PowerPC threat either

In the continuing death march of Spectre side-channel variants for stealing data, all of the known attacks thus far have relied upon code running locally on the computer (so don't run sketchy programs, which have much better ways of pwning your Power Mac than slow and only occasionally successful data leaks). As you'll recall, it is possible for Spectre to succeed on the G5 and 7450 G4e, but not on the G3 and 7400.

The next generation is making Spectre go remote, and while long hypothesized it was never demonstrated until the newest, uh, "advance" called NetSpectre (PDF). The current iteration comes in two forms.

The first and more conventional version is like Spectre in that it relies on CPU cache timing. A victim application would have to have something called a "leak gadget," similar to the one in Spectre where network-facing code processes some network packet with a condition that's usually true and sets a flag based on a data bit of interest in memory. The processor, after enough training by the attacker, then is induced to mispredict, which means the flag is now in the cache even though it never observably changed. This could be done as with the example in the paper, where an attacker sends packets with multiple normal bitstream lengths, training the predictor, and then suddenly sends one with an abnormal or out-of-bound one. The flag isn't actually set, but the misprediction caused it to be loaded into the CPU cache. Later on, the application executes a "transmit gadget" that uses that flag to do a network-observable operation. The flag is in the cache, so the transmit gadget runs just a little bit faster, and the attacker can infer that data bit.

This sounds very slow and error-prone, and it is. In fact, it would be even worse on our slower systems: besides the fact that it presupposes the machine is vulnerable to Spectre in the first place (G3 and 7400 systems don't seem to be), we would generate packets much slower than a modern system, meaning the attacker would have to wait even longer to differentiate a response and the difference between the flag being and not being in the cache is likely to be drowned out by the other code that needs to execute to generate a network response. Looking at the histogram for the ARM core they tested, which is more comparable to the PowerPC than an Intel CPU, there is substantial overlap between the '1' and '0'; if network latency intervenes, it could take literally millions of measurements to extract even a single bit. And that's assuming the attacker knows enough about the innards of your network-facing application (like TenFourFox, or what have you) to even know the memory location they're looking for. Even with that sizeable advantage, even when attacking a far faster computer over a local network, it took 30 minutes for the researchers to exfiltrate just a single byte of data. Under the most optimal conditions for such an attack, a Quad G5 would probably require several times longer; a 7450 would take longer still.

The researchers, however, recognized this and looked for other kinds of network-observable side channels that could be faster to work with than the CPU cache. The vast majority of modern CPUs these days have some sort of SIMD instruction set for working on big chunks of data at once. We have the 128-bit AltiVec (VMX) in Power Mac land on G4s and G5s, for example, and later Power ISA chips like the POWER9 have an extension called VSX; Intel for its part historically offered MMX and the SSE series of instructions all the way up to things like AVX2. AltiVec and VSX are pretty well-designed and reasonably power-efficient extensions but only work on 128 bits of data at once, whereas AVX2 was extended to 256 (AVX-512 even supports 512-bit registers). Intel's larger SIMD implementations require more power to run and the processor actually turns off the circuitry operating on the upper 128 bits of its AVX vector registers when they aren't needed. With that crucial bit of knowledge you can probably write the end of this paragraph already, but turning on the upper 128 bits is not instantaneous and can incur a noticeable penalty on execution if the upper bits aren't already activated. If you can get the processor to speculatively execute an AVX2 instruction operating on the upper bits based on the data bit of interest, you can then infer from how quickly that instruction executed what the data bit was, the execution time itself inferred from a later network-visible operation that also uses the AVX2 upper unit. The AVX2 upper unit cycles on and off with roughly a 1ms latency, an eternity in computing, but it requires very few network measurements to distinguish bits and reduces the time to exfiltrate a byte to around 8 minutes in the paper.

No PowerPC chip used in any Power Mac behaves in this fashion, even with AltiVec instructions. The G3 doesn't have AltiVec (duh), and the AltiVec units in the 7400/G5 (they use similar designs) and the 7450 are always active. AltiVec instructions weren't implemented on "big POWER" until the POWER6, and even for the POWER6 through POWER9, I can't find anything in IBM's technical documentation that says any chip-internal functional unit, whether FPU, LSU, vector unit or otherwise, is dynamically powered down when not in use.

I think we've got bigger things to worry about than this.

OverbiteNX is now available from Mozilla Add-Ons for beta testing

OverbiteNX, a successor to OverbiteFF which allows Firefox to continue to access legacy resources in Gopher in the brave courageous new world of WebExtensions, is now in public beta. Unlike the alpha test, which required you to download the repo and install the extension using add-on debugging, OverbiteNX is now hosted on Mozilla Add-Ons.

Because WebExtensions still doesn't have a TCP sockets API, nor a spec, OverbiteNX uses its bespoke Onyx native component to do network operations. Onyx is written in open-source portable C with no dependencies and is available in pre-built binaries for macOS 10.12+ and Windows (or get the repo and build it yourself on almost any POSIX system).

To try OverbiteNX, install Onyx from the links above, and then install the extension from Mozilla Add-ons. If you use(d) OverbiteWX, which is the proxy-based strict WebExtensions add-on, please disable it as it may conflict. Copious debugging output is emitted to the browser console for this test version. If you file an issue (or better still a pull request) on Github, please include the output so that we can see the execution trace.

Pro tip: sleep's good for your Power Mac

Now that I'm alternating between two daily drivers (my Quad G5 and my Talos II), the Quad G5 sleeps fairly reliably with the Sonnet USB-FireWire combo card out once I move the KVM focus off it. If I don't do that, the KVM detects the G5 has slept and moves the focus automatically away from it, which the G5 detects as USB activity, and exits sleep. The solution is a little AppleScript that waits a few seconds for me to switch the KVM and then tells the Finder to snooze. The Talos II doesn't sleep yet but I'll be interested to see if later firmware updates support that.

But sleeping the G5 has unquestionably been a good thing. Not only does it prolong its life by reducing heat (another plus in summer) as well as saving a substantial amount of energy (around 20W sleeping versus 200-250W running), but sleeping also can speed up TenFourFox. If you have lots of tabs open and those tabs are refreshing their data or otherwise running active content, then this contributes to a greater need for garbage collection and this will slow down your user experience as this overhead accumulates. (This is why running TenFourFox from a "fresh" start is much faster than when it's been chugging away for awhile.) It's possible to "pause" TenFourFox to a certain extent but the browser really isn't tested this way and may not behave properly when this is done. Sleeping the Power Mac pauses everything, so the cruft in memory that garbage collection has to clean out doesn't pile up while you're not using the machine, and everything comes back up in sync.

A whole lot of stuff has landed for TenFourFox FPR9. More about that when the beta is out, which I'm hoping to do by the middle-end of July.

Another one bites the Rust

And another one gone, and another one gone (capitalization sic):

As Herwig Bauernfeind from Bitwise Works made clear in his presentation he gave at Warpstock 2018 Toronto, Firefox for OS/2 is on its way out for OS/2 after version 52 ESR. The primary reason is because Firefox is switching to RUST. Rust is a general purpose programming language sponsored by Mozilla Research. It is unlikely that RUST will ever be ported to OS/2.

Rust was the primary reason we dropped source parity for TenFourFox also (though there were plenty of other reasons such as changes to the graphics stack, the hard requirement for Skia, Electrolysis and changes to ICU; all of this could have been worked around, but with substantial difficulty, and likely with severe compromises). Now that Firefox 52ESR, the last ESR to not require Rust support, is on its last legs, this marks the final end of "Warpzilla" and Firefox on OS/2. SPARC (and apparently Solaris in general) doesn't have rustc or cargo support either, so this is likely the end of Firefox on any version of Solaris as well. Yes, I use Firefox on my Sun Ultra-3 laptop with Solaris 10. There are probably other minor platforms just hanging on that will wink out and disappear which I haven't yet discovered.

Every platform that dies is a loss to the technical diversity of the Mozilla community, no matter how you choose to put a happy face on it.

If you were trying to get a web browser up on a new platform these days, much as it makes me sick to say it, you'd probably be better off with WebKit rather than wrestle with Rust. Or NetSurf, despite its significant limitations, though I love the fact the project even exists. At least there's Rust for the various forms of PowerPC on Linux, including 64-bit and little-endian, so the Talos II can still run Firefox.

With FPR9 TenFourFox will switch to backporting security updates from Firefox 60ESR, though any last-minute chemspills to 52ESR will of course be reviewed promptly.

UPDATE 7/5: Someone in the discussion on Hacker News found that at least $12,650 was raised by the OS/2 community, and they're going to port a Qt based browser, which means ... WebKit. I told you so.

Ad-blocker-blockers hit a new low. What's the solution?

It may be the wrong day to slam the local newspapers, but this was what greeted me trying to click through to a linked newspaper article this morning on Firefox Android. The link I was sent was from the Riverside Press-Enterprise, but this appears to be throughout the entire network of the P-E's owners, the Southern California News Group (which includes the Orange County Register, San Bernardino Sun and Los Angeles Daily News):

That's obnoxious. SCNG is particularly notorious for not being very selective about ads and they tend to be colossally heavy and sometimes invasive; there's no way on this periodically green earth that I'm turning the adblocker off. I click "no thanks." The popover disappears, but what it was covering was this:

That's not me greeking the article so you can't see what article I was reading. The ad-blocker-blocker did it so that a clever user or add-on can't just set the ad-blocker-blocker's popover to display:none or something. The article is now incomprehensible text.

My first reaction is that any possibility I had of actually paying $1 for the 4 week subscription to any SCNG paper just went up in the flames of my great furious wrath (after all, this is a blog s**tpost). The funny part is that TenFourFox's basic adblock actually isn't defeated by this, probably because we're selective about what actually gets blocked and so the ad-blocker-blocker thinks ads are getting through. But our old systems are precisely those that need adblockers because of all the JavaScript (particularly) that modern ad systems lard their impressions up with. Anyway, to read the article I actually ended up looking at it on the G5. There was no way I was going to pay them for engaging in this kind of behaviour.

The second thought I had was, how do you handle this? I'm certainly sympathetic to the view that we need stronger local papers for better local governance, but print ads are a much different beast than the dreck that online ads are. (Yes, this blog has ads. I don't care if you block them or not.) Sure, I could have subscriptions to all the regional papers, or at least the ones that haven't p*ssed me off yet, but then I have to juggle all the memberships and multiple charges and that won't help me read papers not normally in my catchment area. I just want to click and read the news, just like I can anonymously pick up a paper and read it at the bar.

One way to solve this might be to have revenue sharing arrangements between ISPs and papers. It could be a mom-and-pop ISP and the local paper, if any of those or those still exist, or it could be a large ISP and a major national media group. Users on that ISP get free access (as a benefit of membership even), the paper gets a piece. Everyone else can subscribe if they want. This kind of thing already exists on Apple TV devices, after all: if I buy the Spectrum cable plan, I get those channels free on Apple TV over my Spectrum Internet access, or I pay if I don't. Why couldn't newspapers work this way?

Does net neutrality prohibit this?