Sunday, September 11, 2016

Ars Technica's notes from the OS 9 underground

Richard Moss has published his excellent and very comprehensive state of the Mac OS 9 userbase in Ars Technica. I think it's a detailed and very evenhanded summary of why there are more people than you'd think using a (now long obsolete) operating system that's still maintains more utility than you'd believe.

Naturally much of my E-mail interview with him could not be used in the article (I expected that) and I think he's done a fabulous job balancing those various parts of the OS 9 retrocomputing community. Still, there are some things I'd like to see entered into posterity publicly from that interview and with his permission I'm posting that exchange here.

Before doing so, though, just a note to Classilla users. I do have some work done on a 9.3.4 which fixes some JavaScript bugs, has additional stelae for some other site-specific workarounds and (controversially, because this will impact performance) automatically fixes screen update problems with many sites using CSS overflow. (They still don't layout properly, but they will at least scroll mostly correctly.) I will try to push that out as a means of keeping the fossil fed. TenFourFox remains my highest priority because it's the browser I personally dogfood 90% of the time, but I haven't forgotten my roots.

The interview follows. Please pardon the hand-conversion to HTML; I wrote this in plain text originally, as you would expect no less from me. This was originally written in January 2016, and, for the record, on a 1GHz iMac G4.

***

Q. What's your motivation for working on Classilla (and TenFourFox, but I'm mostly interested in Classilla for this piece)?

A. One of the worst things that dooms otherwise functional hardware to apparent obsolescence is when "they can't get on the Internet." That's complete baloney, of course, since they're just as capable of opening a TCP socket and sending and receiving data now as they were then (things like IPv6 on OS 9 notwithstanding, of course). Resources like Gopherspace (a topic for another day) and older websites still work fine on old Macs, even ones based on the Motorola 680x0 series.

So, realistically, the problem isn't "the Internet" per se; some people just want to use modern websites on old hardware. I really intensely dislike the idea that the ability to run Facebook is the sole determining factor of whether a computer is obsolete to some people, but that's the world we live in now. That said, it's almost uniformly a software issue. I don't see there being any real issues as far as hardware capability, because lots of people dig out their old P3 and P4 systems and run browsers on them for light tasks, and older G4 and G3 systems (and even arguably some 603 and 604s) are more than comparable.

Since there are lots of x86 systems, there are lots of people who want to do this, and some clueful people who can still get it to work (especially since operating system and toolchain support is still easy to come by). This doesn't really exist for architectures out of the mainstream like PowerPC, let alone for a now almost byzantine operating system like Mac OS 9, but I have enough technical knowledge and I'm certifiably insane and by dumb luck I got it to work. I like these computers and I like the classic Mac OS, and I want them to keep working and be "useful." Ergo, Classilla.

TenFourFox is a little more complicated, but the same reason generally applies. It's a bit more pointed there because my Quad G5 really is still my daily driver, so I have a lot invested in keeping it functional. I'll discuss this more in detail at the end.

Q. How many people use Classilla?

A. Hard to say exactly since unlike TenFourFox there is no automatic checkin mechanism. Going from manual checkins and a couple back-of-the-napkin computations from download counts, I estimate probably a few thousand. There's no way to know how many of those people use it exclusively, though, which I suspect is a much smaller minority.

Compare this with TenFourFox, which has much more reliable numbers; the figure, which actually has been slowly growing since there are no other good choices for 10.4 and less and less for 10.5, has been a steady 25,000+ users with about 8,000 checkins occurring on a daily basis. That 30% or so are almost certainly daily drivers.

Q. Has it been much of a challenge to build a modern web browser for OS 9? The problems stem from more than just a lack of memory and processing speed, right? What are there deeper issues that you've had to contend with?

A. Classilla hails as a direct descendant of the old Mozilla Suite (along with SeaMonkey/Iceweasel, it's the only direct descendant still updated in any meaningful sense), so the limitations mostly come from its provenance. I don't think anyone who worked on the Mac OS 9 compatible Mozilla will dispute the build system is an impressive example of barely controlled disaster. It's actually an MacPerl script that sends AppleEvents to CodeWarrior and MPW Toolserver to get things done (primarily the former, but one particularly problematic section of the build requires the latter), and as an example of its fragility, if I switch the KVM while it's trying to build stubs, it hangs up and I usually have to restart the build. There's a lot of hacking to make it behave and I rarely run the builder completely from the beginning unless I'm testing something. The build system is so intimidating few people have been able to replicate it on their computers, which has deterred all but the most motivated (or masochistic) contributors. That was a problem for Mozilla too back in the day, I might add, and they were only too glad to dump OS 9 and move to OS X with Mozilla 1.3.

Second, there's no Makefiles, only CodeWarrior project files (previously it actually generated them on the fly from XML templates, but I put a stop to that since it was just as iffy and no longer necessary). Porting code en masse usually requires much more manual work for that reason, like adding new files to targets by hand and so on, such as when I try to import newer versions of libpng or pieces of code from NSS. This is a big reason why I've never even tried to take entire chunks of code like layout/ and content/ even from later versions of the Suite; trying to get all the source files set up for compilation in CodeWarrior would be a huge mess, and wouldn't buy me much more than what it supports now. With the piecemeal hacks in core, it's already nearly to Mozilla 1.7-level as it is (Suite ended with 1.7.13).

Third is operating system support. Mozilla helpfully swallows up much of the ugly guts in the Netscape Portable Runtime, and NSPR is extremely portable, a testament to its good design. But that doesn't mean there weren't bugs and Mac OS 9 is really bad at anything that requires multithreading or multiprocessing, so some of these bugs (like a notorious race condition in socket handling where the socket state can change while the CPU is busy with something else and miss it) are really hard to fix properly. Especially for Open Transport networking, where complex things are sometimes easy but simple things are always hard, some folks (including Mozilla) adopted abstraction layers like GUSI and then put NSPR on top of the abstraction layer, meaning bugs could be at any level or even through subtleties of their interplay.

Last of all is the toolchain. CodeWarrior is pretty damn good as a C++ compiler and my hat is off to Metrowerks for the job they did. It had a very forward-thinking feature set for the time, including just about all of C++03 and even some things that later made it into C++11. It's definitely better than MPW C++ was and light-years ahead of the execrable classic Mac gcc port, although admittedly rms' heart was never in it. Plus, it has an outstanding IDE even by modern standards and the exceptional integrated debugger has saved my pasty white butt more times than I care to admit. (Debugging in Macsbug is much like walking in a minefield on a foggy morning with bare feet: you can't see much, it's too easy to lose your footing and you'll blow up everything to smithereens if you do.) So that's all good news and has made a lot of code much more forward-portable than I could ever have hoped for, but nothing's ever going to be upgraded and no bugs will ever be fixed. We can't even fix them ourselves, since it's closed source. And because it isn't C++11 compliant, we can forget about pulling in substantially more recent versions of the JavaScript interpreter or realistically anything else much past Gecko 2.

Some of the efficiencies possible with later code aren't needed by Classilla to render the page, but they certainly do make it slower. OS 9 is very quick on later hardware and I do my development work on an Power Mac G4 MDD with a Sonnet dual 1.8GHz 7447A upgrade card, so it screams. But that's still not enough to get layout to complete on some sites in a timely fashion even if Classilla eventually is able to do it, and we've got no JIT at all in Classilla.

Mind you, I find these challenges stimulating. I like the feeling of getting something to do tasks it wasn't originally designed to do, sort of like a utilitarian form of the demoscene. Constraints like these require a lot of work and may make certain things impossible, so it requires a certain amount of willingness to be innovative and sometimes do things that might be otherwise unacceptable in the name of keeping the port alive. Making the browser into a series of patches upon patches is surely asking for trouble, but there's something liberating about that level of desperation, anything from amazingly bad hacks to simply accepting a certain level of wrong behaviour in one module because it fixes something else in another to ruthlessly saying some things just won't be supported, so there.

Q. Do you get much feedback from people about the browser? What sorts of things do they say? Do you hear from the hold-outs who try to do all of their computing on OS 9 machines?

A. I do get some. Forcing Classilla to preferring mobile sites actually dramatically improved its functionality, at least for some period of time until sites starting assuming everyone was on some sufficiently recent version of iOS or Android. That wasn't a unanimously popular decision, but it worked pretty well, at least for the time. I even ate my own dogfood and took nothing but an OS 9 laptop with me on the road periodically (one time I took it to Leo Laporte's show back in the old studio, much to his amazement). It was enough for E-mail, some basic Google Maps and a bit of social media.

Nowadays I think people are reasonable about their expectations. The site doesn't have to look right or support more than basic functionality, but they'd like it to do at least that. I get occasional reports from one user who for reasons of his particular disability cannot use OS X, and so Classilla is pretty much his only means of accessing the Web. Other people don't use it habitually, but have some Mac somewhere that does some specific purpose that only works in OS 9, and they'd like a browser there for accessing files or internal sites while they work. Overall, I'd say the response is generally positive that there's something that gives them some improvement, and that's heartening. Open source ingrates are the worst.

The chief problem is that there's only one of me, and I'm scrambling to get TenFourFox 45 working thanks to the never-ending Mozilla rapid release treadmill, so Classilla only gets bits and pieces of my time these days. That depresses me, since I enjoy the challenge of working on it.

Q. What's your personal take on the OS 9 web browsing experience?

A. It's ... doable, if you're willing to be tolerant of the appearance of pages and use a combination of solutions. There are some other browsers that can service this purpose in a limited sense. For example, the previous version of iCab on classic Mac is Acid2 compliant, so a lot of sites look better, but its InScript JavaScript interpreter is glacial and its DOM support is overall worse than Classilla's. Internet Explorer 5.1 (and the 5.5 beta, if you can find it) is very fast on those sites it works on, assuming you can find any. At least when it crashes, it does that fast too! Sometimes you can even get Netscape 4.8 to be happy with them or at least the visual issues look better when you don't even try to render CSS. Most times they won't look right, but you can see what's going on, like using Lynx.

Unfortunately, none of those browsers have up-to-date certificate stores or ciphers and some sites can only be accessed in Classilla for that reason, so any layout or speed advantages they have are negated. Classilla has some other tricks to help on those sites it cannot natively render well itself. You can try turning off the CSS entirely; you could try juggling the user agent. If you have some knowledge of JavaScript, you can tell Classilla's Byblos rewrite module to drop or rewrite problematic portions of the page with little snippets called stelae, basically a super-low-level Greasemonkey that works at the HTML and CSS level (a number of default ones are now shipped as standard portions of the browser).

Things that don't work at all generally require DOM features Classilla does not (yet) expose, or aspects of JavaScript it doesn't understand (I backported Firefox 3's JavaScript to it, but that just gives you the syntax, not necessarily everything else). This aspect is much harder to deal with, though some inventive users have done it with limited success on certain sites.

You can cheat, of course. I have Virtual PC 6 on my OS 9 systems, and it is possible (with some fiddling in lilo) to get it to boot some LiveCDs successfully -- older versions of Knoppix, for example, can usually be coaxed to start up and run Firefox and that actually works. Windows XP, for what that's worth, works fine too (I would be surprised if Vista or anything subsequent does, however). The downside to this is the overhead is a killer on laptops and consumes lots of CPU time, and Linux has zero host integration, but if you're able to stand it, you can get away with it. I reserved this for only problematic sites that I had to access, however, because it would bring my 867MHz TiBook to its knees. The MDD puts up with this a lot better but it's still not snappy.

If all this sounds like a lot of work, it is. But at least that makes it possible to get the majority of Web sites functional to some level in OS 9 (and in Classilla), at least one way or another, depending on how you define "functional." To that end I've started focusing now on getting specific sites to work to some personally acceptable level rather than abstract wide-ranging changes in the codebase. If I can't make the core render it correctly, I'll make some exceptions for it with a stele and ship that with the browser. And this helps, but it's necessarily centric to what I myself do with my Mac OS 9 machines, so it might not help you.

Overall, you should expect to do a lot of work yourself to make OS 9 acceptable with the modern Web and you should accept the results are at best imperfect. I think that's what ultimately drove Andrew Cunningham up the wall.

I'm going to answer these questions together:

Q1. How viable do you think OS 9 is as a primary operating system for someone today? How viable is it for you?
[...]
Q2. What do you like about using older versions of Mac OS (in this case, I'm talking in broad terms - so feel free to discuss OS X Tiger and PPC hardware as well)? Why not just follow the relentless march of technology? (It's worth mentioning here that I actually much prefer the look and feel of classic MacOS and pre-10.6 OS X, but for a lot of my own everyday computing I need to use newer, faster machines and operating systems.)

A. I'm used to a command line and doing everything in text. My Mac OS 9 laptop has Classilla and MacSSH on it. I connect to my home server for E-mail and most other tasks like Texapp for command-line posting to App.net, and if I need a graphical browser, I've got one. That covers about 2/3rds of my typical use case for a computer. In that sense, Mac OS 9 is, at least, no worse than anything else for me. I could use some sort of Linux, but then I wouldn't be able to easily run much of my old software (see below). If I had to spend my time in OS 9 even now, with a copy of Word and Photoshop and a few other things, I think I could get nearly all of my work done, personally. There is emulation for the rest. :)

I will say I think OS 9 is a pleasure to use relative to OS X. Part of this is its rather appalling internals, which in this case is necessity made virtue; I've heard it said OS 9 is just a loose jumble of libraries stacked under a file browser and that's really not too far off. The kernel, if you can call it that, is nearly non-existent -- there's a nanokernel, but it's better considered as a primitive hypervisor. There is at best token support for memory protection and some multiprocessing, but none of it is easy and most of it comes with severe compromises. But because there isn't much to its guts, there's very little between you and the hardware. I admit to having an i7 MBA running El Crapitan, and OS 9 still feels faster. Things happen nearly instantaneously, something I've never said about any version of OS X, and certain classes of users still swear by its exceptionally low latency for applications such as audio production. Furthermore, while a few annoyances of the OS X Finder have gradually improved, it's still not a patch on the spatial nature of the original one, and I actually do like Platinum (de gustibus non disputandum, of course). The whole user experience feels cleaner to me even if the guts are a dog's breakfast.

It's for that reason that, at least on my Power Macs, I've said Tiger forever. Classic is the best reason to own a Power Mac. It's very compatible and highly integrated, to the point where I can have Framemaker open and TenFourFox open and cut and paste between them. There's no Rhapsody full-screen blue box or SheepShaver window that separates me from making Classic apps first-class citizens, and I've never had a Classic app take down Tiger. Games don't run so well, but that's another reason to keep the MDD around, though I play most of my OS 9 games on a Power Mac 7300 on the other desk. I've used Macs partially since the late 1980s and exclusively since the mid-late 1990s (the first I owned personally was a used IIsi), and I have a substantial investment in classic Mac software, so I want to be able to have my cake and eat it too. Some of my preference for 10.4 is also aesthetic: Tiger still has the older Mac gamma, which my eyes are accustomed to after years of Mac usage, and it isn't the dreary matte grey that 10.5 and up became infested with. These and other reasons are why I've never even considered running something like Linux/PPC on my old Macs.

Eventually it's going to be the architecture that dooms this G5. This Quad is still sufficient for most tasks, but the design is over ten years old, and it shows. Argue the PowerPC vs x86 argument all you like, but even us PPC holdouts will concede the desktop battle was lost years ago. We've still got a working gcc and we've done lots of hacking on the linker, but Mozilla now wants to start building Rust into Gecko (and Servo is, of course, all Rust), and there's no toolchain for that on OS X/ppc, so TenFourFox's life is limited. For that matter, so is general Power Mac software development: other than freaks like me who still put -arch ppc -arch i386 in our Makefiles, Universal now means i386/x86_64, and that's not going to last much longer either. The little-endian web (thanks to asm.js) even threatens that last bastion of platform agnosticism. These days the Power Mac community lives on Pluto, looking at a very tiny dot of light millions of miles away where the rest of the planets are.

So, after TenFourFox 45, TenFourFox will become another Classilla: a fork off Gecko for a long-abandoned platform, with later things backported to it to improve its functionality. Unlike Classilla it won't have the burden of six years of being unmaintained and the toolchain and build system will be in substantially better shape, but I'll still be able to take the lessons I've learned maintaining Classilla and apply it to TenFourFox, and that means Classilla will still live on in spirit even when we get to that day when the modern web bypasses it completely.

I miss the heterogeneity of computing when there were lots of CPUs and lots of architectures and ultimately lots of choices. I think that was a real source of innovation back then and much of what we take for granted in modern systems would not have been possible were it not for that competition. Working in OS 9 reminds me that we'll never get that diversity back, and I think that's to our detriment, but as long as I can keep that light on it'll never be completely obsolete.

***

Saturday, September 10, 2016

TenFourFox 45.4.0 available (plus: priorities for feature parity and down with Dropbox)

With the peerless level of self-proclaimed courageousness necessary to remove a perfectly good headphone jack for a perfectly stupid reason (because apparently no one at Apple charges their phone and listens to music at the same time), TenFourFox 45.4.0 is released (downloads, hashes, release notes). This will be the final release in the beta cycle and assuming no critical problems will become the public official release sometime Monday Pacific time. Localizations will be frozen today and uploaded to SourceForge sometime this evening or tomorrow, so get them in ASAP.

The major change in this release is additional tweaking to the MediaSource implementation and I'm now more comfortable with its functioning on G4 systems through a combination of some additional later patches I backported and adjusting our own hacks to not only aggressively report the dropped frames but also force rebuffering if needed. The G4 systems now no longer seize and freeze (and, occasionally, fatally assert) on some streams, and the audio never becomes unsynchronized, though there is some stuttering if the system is too overworked trying to keep the video and audio together. That said, I'm going to keep MediaSource off for 45.4 so that there will be as little unnecessary support churn as possible while you test it (if you haven't already done so, turn media.mediasource.enabled to true in about:config; do not touch the other options). In 45.5, assuming there are no fatal problems with it (I don't consider performance a fatal flaw, just an important one), it will be the default, and it will be surfaced as an option in the upcoming TenFourFox-specific preference pane.

However, to make the most of MediaSource we're going to need AltiVec support for VP9 (we only have it for VP3 and VP8). While upper-spec G5 systems can just power through the decoding process (though this might make hi-def video finally reasonable on the last generation machines), even high-spec G4 systems have impaired decoding due to CPU and bus bandwidth limitations and the low-end G4 systems are nearly hopeless at all but the very lowest bitrates. Officially I still have a minimum 1.25GHz recommendation but I'm painfully aware that even those G4s barely squeak by. We're the only ones who ever used libvpx's moldy VMX code for VP8 and kept it alive, and they don't have anything at all for VP9 (just x86 and ARM, though interestingly it looks like MIPS support is in progress). Fortunately, the code was clearly written to make it easier for porters to hand-vectorize and I can do this work incrementally instead of having to convert all the codec pieces simultaneously.

Interestingly, even though our code now faithfully and fully reports every single dropped frame, YouTube doesn't seem to do anything with this information right now (if you right-click and select "Stats for nerds" you'll see our count dutifully increase as frames are skipped). It does downshift for network congestion, so I'm trying to think of a way to fool it and make dropped frames look like a network throughput problem instead. Doing so would technically be a violation of the spec but you can't shame that which has no shame and I have no shame. Our machines get no love from Google anyway so I'm perfectly okay with abusing their software.

I have the conversion to platform codec of our minimp3 decoder written as a first draft, but I haven't yet set that up or tested it, so this version still uses the old codec wrapper and still has the track-shifting problem with Amazon Music. That is probably the highest priority for 45.5 since it is an obvious regression from 38. On the security side, this release also disables RTCPeerConnection to eliminate the WebRTC IP "leak" (since I've basically given up on WebRTC for Power Macs). You can still reenable it from about:config as usual.

The top three priorities for the next couple versions (with links to the open Github issues) are, highest first, fixing Amazon Music, AltiVec VP9 codepaths and the "little endian typed array" portion of IonPower-NVLE to fix site compatibility with little-endian generated asm.js. Much of this work will proceed in parallel and the idea is to have a beta 45.5 for you to test them in a couple weeks. Other high priority items on my list to backport include allowing WebKit/Blink to eat the web supporting certain WebKit-prefixed properties to make us act the same as regular Firefox, support for ChaCha20+Poly1305, WebP images, expanded WebCrypto support, the "NV" portion of IonPower-NVLE and certain other smaller-scope HTML/CSS features. I'll be opening tracking issues for these as they enter my worklist, but I have not yet determined how I will version the browser to reflect these backported new features. For now we'll continue with 45.x.y while we remain on 45ESR and see where we end up.

As we look into the future, though, it's always instructive to compare it with the past. With the anticipation that even Google Code's Archive will be flushed down the Mountain View memory hole (the wiki looks like it's already gone, but you can get most of our old wikidocs from Github), I've archived 4.0b7, 4.0.3, 8.0, 10.0.11, 17.0.11 and Intel 17.0.2 on SourceForge along with their corresponding source changesets. These Google Code-only versions were selected as they were either terminal (quasi-)ESR releases or have historical or technical relevance (4.0b7 was our first beta release of TenFourFox ever "way back" in 2010, 8.0 was the last release that was pure tracejit which some people prefer, and of course Intel 17.0.2 was our one and so far only release on Intel Macs). There is no documentation or release notes; they're just there for your archival entertainment and foolhardiness. Remember that old versions run an excellent chance of corrupting your profile, so start them up with one you can throw away.

Finally, a good reason to dump Dropbox (besides the jerking around they give those of you trying to keep the PowerPC client working) is their rather shameful secret unauthorized abuse of your Mac's accessibility framework by forging an entry in the privacy database. (Such permissions allow it to control other applications on your Mac as if it were you at the user interface. The security implications of that should be immediately obvious, but if they're not, see this discussion.) The fact this is possible at all is a bug Apple absolutely must fix and apparently has in macOS Sierra, but exploiting it in this fashion is absolutely appalling behaviour on Dropbox's part because it won't even let you turn it off. To their credit they're taking their lumps on Hacker News and TechCrunch, but accepting their explanation of "only asking for privileges we use" requires a level of trust that frankly they don't seem worthy of and saying they never store your administrator password is a bit disingenuous when they use administrative access to create a whole folder of setuid binaries -- they don't need your password at that point to control the system. Moreover, what if there were an exploitable security bug in their software?

Mind you, I don't have a problem with apps requesting that access if I understand why and the request isn't obfuscated. As a comparison, GOG.com has a number of classic DOS games I love that were ported for DOSBox and work well on my MacBook Air. These require that same accessibility access for proper control of input methods. Though I hope they come up with a different workaround eventually, the GOG installer does explain why and does use the proper APIs for requesting that privilege, and you can either refuse on the spot or disable it later if you decide you're not comfortable with it. That's how it's supposed to work, but that's not what Dropbox did, and they intentionally hid it and the other inappropriate system-level things they were sneaking through. Whether out of a genuine concern for user experience or just trying to get around what they felt was an unnecessary security precaution, it's not acceptable and it's potentially exploitable, and they need to answer for that.

Watch for 45.4 going final in a couple days, and hopefully a 45.5 beta in a couple weeks.

Tuesday, September 6, 2016

System7SoonToBeYesterday

One of my periodic "drop by now and then" sites is System7Today, extolling the virtues of System 7 to those people still rocking 60x-series Power Macs (that's Mac OS 7.0 through 7.6.1 for those of you who only know Mac OS as starting with a lower-case m, and also, get off my lawn), and I was disheartened to see that the System7Today forums are going read-only. I regretfully understand his reasoning though one wonders what will happen to the rest of the site. It hasn't changed in years, but it hasn't had to, and I love the frozen-in-time mockery of the Apple front page from at least a decade or so prior.

The only Macs I have still running System 7 are all running 7.1 (not counting the NetBSD 68K systems, which just use System 7 as a bootloader): one is my IIci with a 50MHz DayStar '030 and a MacIvory Lisp card, another is my recapped SE/30 looking for a job to do, and the last is my super-cute Mystic Color Classic. I like 7.1 a lot more than 7.5 or 7.6, and you can transplant lots of the 7.5 CDEVs and such back to 7.1 for a slimmer but still feature-filled experience. That said, I have to confess that I jumped to OS 8, and then OS 9, whenever I got the chance on my Power Macs. Part of this was that I upgraded those systems aggressively -- all of my Old World Power Macs in regular use have G3 or G4 upgrade cards, including my beloved PowerBook 1400, so they all run 9.1 or 9.2.2 -- and part of it is to run Classilla, but the biggest reason was just that OS 8 looked nicer and felt better, and 8.1 and 8.6 were still pretty speedy. I still run OS 8 on my PowerBook 2300c and Quadra 800.

But still, nostalgia dies hard. (No doubt being a TenFourFox user, you'd empathize.) While I've got lots of classic System 7-era software backed up for posterity on the Floodgap gopher server, it just doesn't have System7Today's throwback vibe and playful attitude. I'll miss it dearly pending its inevitable slow-motion decommissioning, most of all because it remains a great example of doing the most you can with the little you've got. Typing this blog post on a 2002-vintage 1GHz iMac G4, us PowerPC holdouts are still living in that spirit today.

Watch for 45.4 later this week(end).

Thursday, September 1, 2016

Talos moves to CrowdSupply

It looks like we've cleared the hump: Raptor Engineering tells me that Talos, the upcoming POWER8 workstation and probably the most powerful non-x86 workstation-class system you can buy, has enough interest to move to a formal CrowdSupply launch. Sign up and stay tuned. I've been still saving my pennies and I'm definitely buying.

Friday, August 26, 2016

TenFourFox 45.3.0b2 available

TenFourFox 45.3 beta 2 is now available (downloads, hashes, release notes). The issue with video frame skipping gone awry seems to be due to Mozilla's MediaSource implementation, which is (charitably) somewhat incomplete on 45 and not well-tested on systems of our, uh, vintage. Because audio is easier to decode than video the audio decoder thread will already have a speed advantage, but in addition the video decoder thread appears to only adjust itself for the time spent in decoding, not the time spent pushing the frames to the compositor or blitting them to the screen (which are also fairly costly in our software-rendered world). A frame that's simple to decode but requires a lot of painting will cause the video thread to get further and further behind queueing frames that the compositor can't keep up with. Even if you keep skipping keyframes, you'll end up more than one keyframe behind and you'll never catch up, and when this occurs the threads irrevocably go out of sync. The point at which this happens for many videos is almost deterministic and correlates well with time spent in compositing and drawing; you can invariably trigger it by putting the video into full screen mode which dramatically increases the effort required to paint frames. In a few cases the system would just give up and refuse to play further (hitting an assertion in debug mode).

After a few days of screaming and angry hacking (backporting some future bug fixes, new code for dumping frames that can never be rendered, throwing out the entire queue if we get so far behind we'll never reasonably catch up) I finally got the G5 to be able to hold its own in Reduced Performance but it's just too far beyond the G4 systems to do so. That's a shame since when it does work it appears to work pretty well. Nevertheless, for release I've decided to turn MediaSource off entirely and it now renders mostly using the same pipeline as 38 (fortunately, the other improvements to pixel pushing do make this quicker than 38 was; my iBook G4/1.33 now maintains a decent frame rate for most standard definition videos in Automatic Performance mode). If the feature becomes necessary in the future I'm afraid it will only be a reasonable option for G5 machines in its current state, but at least right now disabling MediaSource is functional and sufficient for the time being to get video support back to where it was.

The issue with the Amazon Music player is not related to or fixed by this change, however. I'm thinking that I'll have to redo our minimp3 support as a platform MP3 decoder so that we use more of Mozilla's internal media machinery; that refactoring is not likely to occur by release, so we'll ship with the old decoder that works most other places and deal with it in a future update. I don't consider this a showstopper unless it's affecting other sites.

Other changes in 45.3 beta 2 are some additional performance improvements. Most of these are trivial (including some minor changes to branching and proper use of single-precision math in the JIT), but two are of note, one backport that reduces the amount of allocation with scrolling and improves overall scrolling performance as a result, and another one I backported that reduces the amount of scanning required for garbage collection (by making garbage collection only run in the "dirty" zones after cycle collection instead of effectively globally). You'll notice that RAM use is a bit higher at any given moment, but it does go back down, and the browser has many fewer random pauses now because much less work needs to be done. Additionally in this beta unsigned extensions should work again and the update source is now set to the XML for 45. One final issue I want to repair before launch is largely server-side, hooking up the TenFourFox help button redirector to use Mozilla's knowledge base articles where appropriate instead of simply redirecting to our admittedly very sparse set of help documents on Tenderapp. This work item won't need a browser update; it will "just work" when I finish the code on Floodgap's server.

At this point barring some other major crisis I consider the browser shippable as 45.4 on time on 13 September, with Amazon Music being the highest priority issue to fix, and then we can start talking about what features we will add while we remain on feature parity. There will also be a subsequent point release of TenFourFoxBox to update foxboxes for new features in 45; stay tuned for that. Meanwhile, we could sure use your help on a couple other things:

  • Right now, our localized languages for the 45 launch are German, Spanish, French and Japanese, with an Italian translation in progress. We can launch with that, but we're losing a few languages there we had with 38. You can help.

  • Even minor releases of the browser get a small bump in support tickets on Tenderapp; a major release (such as 31 to 38, and now 38 to 45) generally brings lots of things out of the woodwork that we've never seen during the beta cycle. Most of these reports are spurious, but sometimes serious bugs have been uncovered even though they looked obviously bogus at first glance, and all of them need to be triaged regardless. If you're a reasonably adept user and you've got a decent understanding of how the browser works, help us help your fellow users by being a volunteer on Tenderapp. There's no minimum time commitment required; any help you can offer is appreciated, even if just temporary. Right now it's just Chris Trusch and me trying to field these reports, which is unsatisfactory to cover all of them (Theo used to but we haven't seen him in awhile). If you'd like to help, post in the comments and we'll make you a "blue name" on Tenderapp too.

That's it for now. See what you think.

Friday, August 19, 2016

TenFourFox 45 beta 2: not yet

So far the TenFourFox 45 beta is doing very well. There have been no major performance regressions overall (Amazon Music notwithstanding, which I'll get to presently) and actually overall opinion is that 45 seems much more responsive than 38. On that score, you'll like beta 2 even more when I get it out: I raided some more performance-related Firefox 46 and 47 patches and backported them too, further improving scrolling performance by reducing unnecessary memory allocation and also substantially reducing garbage collection overhead. There are also some minor tweaks to the JavaScript JIT, including a gimme I should have done long ago where we use single-precision FPU instructions instead of double-precision plus a rounding step. This doesn't make a big difference for most scripts, but a few, particularly those with floating point arrays, will benefit from the reduction in code size and slight improvement in speed.

Unfortunately, while this also makes Amazon Music's delay on starting and switching tracks better, it's still noticeably regressed compared to 38. In addition, I'm hitting an assertion with YouTube on certain, though not all, videos over a certain minimum length; in the release builds it just seizes up at that point and refuses to play the video further. Some fiddling in the debugger indicates it might be related to what Chris Trusch was reporting about frameskipping not working right in 45. If I'm really lucky all this is related and I can fix them all by fixing that root problem. None of these are showstoppers but that YouTube assertion is currently my highest priority to fix for the next beta; I have not solved it yet though I may be able to wallpaper it. I'm aiming for beta 2 this coming week.

On the localization front we have French, German and Japanese translations, the latter from the usual group that operates separately from Chris T's work in issue 42. I'd like to get a couple more in the can before our planned release on or about September 13. If you can help, please sign up.

Wednesday, August 10, 2016

And now for something completely different: what you missed by not going to Vintage Computer Festival West XI

With the exception of this man,
you all missed out by not coming to the reincarnated Vintage Computer Festival West. You had a chance to meet me and my beloved Tomy Tutor (which you can emulate on a Power Mac or Intel Mac). You had a chance to see other inferior exhibits. You didn't. So let me now make you feel bad about what you missed out on.

You could have seen the other great exhibits at the Computer History Museum,

the complete Tomy Tutor family, hosted by my lovely wife and me
that corrupted young minds from a very early age
(for which I won a special award for Most Complete and 2nd place overall in the Microcomputer category),
seen a really big MOS 6502 recreation,
played Maze War,
solved differential equations the old fashioned way,
realized that in Soviet Russia computer uses you,
given yourself terrible eyestrain,
used a very credible MacPaint clone on a Tandy Color Computer,
messed with Steve Chamberlain's mind,
played with an unusual Un*x workstation powered by the similarly unusual (and architecturally troublesome) Intel i860,
marveled at the very first Amiga 1000 (serial #1),
attempted to steal an Apple I,
snuck onto Infinite Loop on the way back south,
and got hustled off by Apple security for plastering "Ready for PowerPC upgrade" stickers all over the MacBooks in the company store.

But you didn't. And now it's too late.*

*Well, you could always come next year.