Sunday, March 31, 2019

TenFourFox FPR14b1 available (now with H.264 video)

TenFourFox Feature Parity Release 14 beta 1 is now available (downloads, hashes, release notes).

I had originally plotted three main features for this release, but getting the urgent FPR13 SPR1 set me back a few days with confidence testing and rebuilds and I have business trips and some vacation time coming up, so I jettisoned the riskiest of the three features (a set of JavaScript updates and a ugly hack to get Github and other sites working fully again) and concentrated on the other two. I'll be looking at that again for FPR15, so more on that later.

Before we get to the marquee features, though, there are two changes which you may not immediately notice. The first is a mitigation for a long-standing issue where some malicious sites keep popping up authentication modals using HTTP Auth. Essentially you can't do anything with the window until the modal is dealt with, so the site just asks for your credentials over and over, ultimately making the browser useless (as a means to make you call their "support line" where they can then social engineer their way into your computer). The ultimate solution is to make such things tab-modal rather than window-modal, but that's involved and sort of out of scope, so we now implement a similar change to what current Firefox does where there is a cap of three Cancels. If you cancel three times, the malicious site is not allowed to issue any more requests until you reload it. No actual data is leaked, assuming you don't type anything in, but it can be a nasty denial of service and it would have succeeded in ruining your day on TenFourFox just as easily as any other Firefox derivative. That said, just avoid iffy sites, yes?

The second change is more fundamental. For Firefox 66 Mozilla briefly experimented with setting a frame rate cap on low-end devices. Surprise, surprise: all of our systems are low-end devices! In FPR13 and prior, TenFourFox would try to push as many frames to the compositor as possible, no matter what it was trying to do, to achieve a 60fps target or better. However, probably none of our computers with the possible exception of high-end G5s were probably achieving 60fps consistently on most modern websites, and the browser would flail trying to desperately keep up. Instead, by setting a cap and enforcing it with software v-sync, frames aren't pushed as often and the browser can do more layout and rendering work per frame. Mozilla selected a 30fps cap, so that's what I selected as an arbitrary first cut. Some sites are less smooth, but many sites now render faster to first paint, particularly pages that do a lot of DOM transforms because now the resulting visual changes are batched. This might seem like an obvious change to make but the numbers had never been proven until then.

Mozilla ultimately abandoned this change in lieu of a more flexible approach with the idle queue, but our older codebase doesn't support that, and we don't have the other issues they encountered anyway because we don't support Electrolysis or APZ. There are two things to look at: we shouldn't have the same scrolling issues because we scroll synchronously, but do report any regressions in default scrolling or obvious changes in scroll rate (include what you're doing the scrolling with, such as the scroll bar, a mouse scroll wheel or your laptop trackpad). The second thing to look at is whether the 30fps frame rate is the right setting for all systems. In particular, should 7400 or G3 be even lower, maybe 15fps? You can change this with layout.frame_rate to some other target frame rate value and restarting the browser. What setting seems to do best on your machine? Include RAM, OS and CPU speed. One other possibility is to look at reducing the target frame rate dynamically based on battery state, but this requires additional plumbing we don't support just yet.

So now the main event: H.264 video support. Olga gets the credit here for the original code, which successfully loads our separately-distributed build of ffmpeg so that we don't run afoul of any licenses including it with the core browser. My first cut of this had issues where the browser ran out of memory on sites that ran lots of H.264 video as images (and believe me, this is not at all uncommon these days), but I got our build of ffmpeg trimmed down enough that it can now load the Vimeo front page and other sites generally without issue. Download the TenFourFox MP4 Enabler for either G4/7450 or G5 (this is a bit of a misnomer since we also use ffmpeg for faster MP3 and VP3 decoding, but I didn't want it confused with Olga's preexisting FFmpeg Enabler), download FPR14b1, run the Enabler to install the libraries and then start FPR14b1. H.264 video should now "just work." However, do note there may be a few pieces left to add for compatibility (for example, Twitter videos used to work and then something changed and now it doesn't and I don't know why, but Imgur, YouTube and Vimeo seem to work fine).

There are some things to keep in mind. While ffmpeg has very good AltiVec support, H.264 video tends to be more ubiquitous and run at higher bitrates, which cancel out the gains; I wouldn't expect dramatic performance improvements relative to WebM and while you may see them in FPR14 relative to FPR13 remember that we now have a frame rate cap which probably makes the decoder more efficient. As mentioned before, I only support G4/7450 (and of those, 1.25GHz and up) and G5 systems; a G4/7400 will have trouble keeping up even with low bitrates and there's probably no hope for G3 systems at all. The libraries provided are very stripped down both for performance and to reduce size and memory pressure, so they're not intended as a general purpose ffmpeg build (in particular, there are no encoders, multiplexers or protocols, some codecs have been removed, and VP8/VP9 are explicitly disabled since our in-tree hyped-up libvpx is faster). You can build your own libraries and put them into the installation location if you want additional features (see the wiki instructions for supported versions and the build process), and you may want to do this for WebM in particular if you want better quality since our build has the loop filter and other postprocessing cut down for speed, but I won't offer support for custom libraries and you'd be on your own if you hit a bug. Finally, the lockout code I wrote when I was running into memory pressure issues is still there and will still cancel all decoding H.264 instances if any one of them fails to get memory for a frame, hopefully averting a crash. This shouldn't happen much now with the slimmer libraries but I still recommend as much RAM as your system can hold (at least 2GB). Oh, and one other thing: foxboxes work fine with H.264!

Now, enjoy some of the Vimeo videos you previously could only watch with the old PopOutPlayer, back when it actually still worked. Here are four of my favourites: Vicious Cycle (PG-13 for pixelated robot blood), Fired On Mars (PG-13 for an F-bomb), Other Half (PG-13 for an F-bomb and oddly transected humans), and Unsatisfying (unsatisfying). I've picked these not only because they're entertaining but also because they run the gamut from hand-drawn animation to CGI to live action and give you an idea of how your system performs. However, I strongly recommend you click the gear icon and select 360p before you start playback, even on a 2005 G5; HD performance is still best summarized as "LOL."

At least one of you asked how to turn it off. Fortunately, if you never install the libraries, it'll never be "on" (the browser will work as before). If you do install them, and decide you prefer the browser without it, you can either delete the libraries from ~/Library/TenFourFox-FFmpeg (stop the browser first just in case it has them loaded) or set media.ffmpeg.enabled to false.

The other, almost anticlimactic change, is that you can now embed JavaScript in your AppleScripts and execute them in browser tabs with run JavaScript. While preparing this blog post I discovered an annoying bug in the AppleScript support, but since no one has reported it so far, it must not be something anyone's actually hitting (or I guess no one's using it much yet). It will be fixed for final which will come out parallel with Firefox 60.7/67 on or about May 14.

Sunday, March 24, 2019

TenFourFox FPR13 SPR1 available

TenFourFox Feature Parity Release 13 Security Parity Release 1 ("FPR13.1") is now available and live (downloads, hashes, release notes). The Pwn2Own vulnerabilities do not work on TenFourFox in their present form (not only because we're PowerPC but also because of our hybrid-endian typed arrays and other differences), but I have determined that TenFourFox-specific variant attacks could be exploitable, so we are patched as well. This should also reduce the risk of crashes from attempts to exploit mainline x86 Firefox.

Meanwhile, H.264 support for TenFourFox FPR14 appears to be sticking. Yes, folks: for the first time you can now play Vimeo and other H.264-only videos from within TenFourFox using sidecar ffmpeg libraries, and it actually works pretty well! Kudos to Olga for the integration code! That said, however, it comes with a couple significant caveats. The first is that while WebM video tends not to occur in large numbers on a given page, H.264 videos nowadays are studded everywhere (Vimeo's front page, Twitter threads, Imgur galleries, etc.) and sometimes try to autoplay simultaneously. In its first iteration this would cause the browser to run out of memory if a large number of higher resolution videos tried to play at once, and sometimes crash when an infallible memory allocation fallibled. Right now there is a lockout in the browser to immediately halt all H.264 decoding if any instance runs out of memory so that the browser can save itself, but this needs a lot more testing to make sure it's solid, and is clearly a suboptimal solution. Remember that we are under unusual memory constraints because of the large amount of stack required for our JIT.

The second caveat with H.264 support is that while the additional AltiVec support in ffmpeg (TenFourFox is compatible with 2.8 and 3.4) makes H.264 decoding faster than WebM, it is not dramatically so, and you should not expect a major jump in video performance. (In fact, quite the opposite on pages like the above.) Because of that, and because I have to build and support ffmpeg library installers now, I am only officially supporting H.264 on G4/7450 and G5 based on the existing 1.25GHz minimum CPU requirement for web video (and you should really have 2GB or more of memory). There will not be an official TenFourFox ffmpeg build for G4/7400 and G3 (or, for that matter, Intel); while you can build it yourself mostly out of the box with Xcode 2.5 and I won't have any block in TenFourFox for user-created libraries, I will provide neither support nor ffmpeg builds for these architectures. Olga's current FFmpeg Enabler does work on 10.4 now and does support 7400 and my future 7450 version will run on a 7400, so early G4 users have a couple options, but either way you would be on your own. Sorry, there are enough complaints about TenFourFox performance already without me making promises of additional functionality I know those systems can't meet.

Back on the good news side, the AppleScript-JavaScript bridge is also complete and working. As a example, consider this script, which actually works in the internal test build:

tell application "TenFourFoxG5"
  tell front browser window
    set URL of current tab to "https://www.google.com/"
    repeat while (current tab is busy)
      delay 1
    end repeat
    tell current tab
      run JavaScript "let f = document.getElementById('tsf');f.q.value='tenfourfox';f.submit();"
    end tell
    repeat while (current tab is busy)
      delay 1
    end repeat
    tell current tab
      run JavaScript "return document.getElementsByTagName('h3')[0].innerText + ' ' + document.getElementsByTagName('cite')[0].innerText"
    end tell
  end tell
end tell

I'll let you ponder what it does until the FPR14 beta comes out, but it should be obvious that this would be great for automating certain tasks in the browser now that you don't have to rely on figuring out how to send the exact UI event anymore: you can just manipulate the DOM of any web page directly from AppleScript. Firefox still can't do that! (Mozilla can port over my code; I'd be flattered.)

The last things to do are a couple security and performance tweaks, and then one more desperate attempt to get Github working. I'm still not sure how feasible the necessary JavaScript hacks will be yet but come hell or high water we're on track for FPR14 beta 1 in early April.

Friday, March 22, 2019

Stand by for urgent security update

Pwn2Own came and went and Firefox fell with it. The __proto__ vulnerability seems exploitable in TenFourFox, though it would require a PowerPC-specific attack to be fully weaponized, and I'm currently evaluating the other bug. Builds ("FPR13 SPR1") including fixes for either or both depending on my conclusions will be issued within the next couple days.

Saturday, March 16, 2019

TenFourFox FPR13 available

TenFourFox Feature Parity Release 13 final is now available for testing (downloads, hashes, release notes). I added Olga's minimp3 patch for correctness; otherwise, there are no additional changes except for several security updates and to refresh the certificate and TLD stores. As usual it will go live Monday evening Pacific time assuming no difficulties.

I have three main updates in mind for TenFourFox FPR14: expanding FPR13's new AppleScript support to allow injecting JavaScript into pages (so that you can drive a web page by manipulating the DOM elements within it instead of having to rely on screen coordinates and sending UI events), adding Olga's ffmpeg framework to enable H.264 video support with a sidecar library (see the previous post for details on the scheme), and a possible solution to allow JavaScript async functions which actually might fix quite a number of presently non-working sites. I'm hopeful that combined with another parser hack this will be enough to restore Github functionality on TenFourFox, but no promises. Unfortunately, it doesn't address the infamous this is undefined problem that continues to plague a number of sites and I still have no good solution for that. These projects are decent-sized undertakings, so it's possible one or two might get pushed to FPR15. FPR14 is scheduled for May 14 with Firefox 67.

Meanwhile, I took a close look at the upcoming Raptor Blackbird at the So Cal Linux Expo 17. If the full big Talos II I'm typing this on is still more green than you can dream, the smaller Blackbird may be just your size to get a good-performing 64-bit Power system free of the lurking horrors in modern PCs at a better price. Check out some detailed board pics of the prototype and other shots of the expo on Talospace. If you're still not ready to jump, I'll be reviewing mine when it arrives hopefully later this spring.

Saturday, March 2, 2019

Another choice for Intel TenFourFox users

Waaaaaaay back when, I parenthetically mentioned in passing an anonymous someone(tm) trying to resurrect the then-stalled Intel port. Since then we now have a periodically updated unofficial and totally unsupported mainline Intel version, but it wasn't actually that someone who was working on it. That someone now has a release, too.

@OlgaTPark's Intel TenFourFox fork is a bit unusual in that it is based on 45.9 (yes, back before the FPR releases began), so it is missing later updates in the FPR series. On the other hand, it does support Tiger (mainline Intel TenFourFox requires at least 10.5), it additionally supports several features not supported by TenFourFox, i.e., by enabling Mozilla features in some of its operating system-specific flavours that are disabled in TenFourFox for reasons of Tiger compatibility, and also includes support for H.264 video with ffmpeg.

H.264 video has been a perennial request which I've repeatedly nixed for reasons of the MPEG LA threatening to remove and purée the genitals of those who would use its patents without a license, and more to the point using ffmpeg in Firefox and TenFourFox probably would have violated the spirit, if not the letter, of the Mozilla Public License. Currently, mainline Firefox implements H.264 using operating system support and the Cisco decoder as an external plugin component. Olga's scheme does much the same thing using a separate component called the FFmpeg Enabler, so it should be possible to implement the glue code in mainline TenFourFox, "allowing" the standalone, separately-distributed enabler to patch in the library and thus sidestepping at least the Mozilla licensing issue. The provided library is a fat dylib with PowerPC and Intel support and the support glue is straightforward enough that I may put experimental support for this mechanism in FPR14.

(Long-time readers will wonder why there is MP3 decoding built into TenFourFox, using minimp3 which itself borrows code from ffmpeg, if I have these objections. There are three simple reasons: MP3 patents have expired, it was easy to do, and I'm a big throbbing hypocrite. One other piece of "OlgaFox" that I'll backport either for FPR13 final or FPR14 is a correctness fix for our MP3 decoder which apparently doesn't trip up PowerPC, but would be good for Intel users.)

Ordinarily I don't like forks using the same name, even if I'm no longer maintaining the code, so that I can avoid receiving spurious support requests or bug reports on code I didn't write. For example, I asked the Oysttyer project to change names from TTYtter after I had ceased maintaining it so that it was clearly recognized they were out on their own, and they graciously did. In this case, though it might be slightly confusing, I haven't requested my usual policy because it is clearly and (better be) widely known that no Intel version of TenFourFox, no matter what version or what features, is supported by me.

On the other hand, if someone used Olga's code as a basis for, say, a 10.5-specific PowerPC fork of TenFourFox enabling features supported in that OS (a la the dearly departed AuroraFox), I would have to insist that the name be changed so we don't get people on Tenderapp with problem reports about it. Fortunately, Olga's release uses the names TenFiveFox and TenSixFox for those operating system-specific versions, and I strongly encourage anyone who wants to do such a Leopard-specific port to follow suit.

Releases can be downloaded from Github, and as always, there is no support and no promises of updates. Do not send support questions about this or any Intel build of TenFourFox to Tenderapp.

Saturday, February 23, 2019

And now for something completely different: The G5 that went to the dark side of the X-Force

My favourite console of all time is still the ahead-of-its-time Sega Dreamcast, but a close second on my list is the PowerPC-based Microsoft Xbox 360. Besides being our favourite architecture, it plays a number of great games (I still think BioShock Infinite is one of the best games ever written, and it could have been even better) and many original Xbox titles, and it's even on topic here as Microsoft shipped Power Mac G5s as development systems to early Xbox 360 developers (the X360 being powered by the Xenon CPU, designed by IBM and using three of the PowerPC PPE cores that also appeared in the Cell processor). There was a famous Anandtech article I remember where at E3 2005 they discovered the "consoles" were props, and what people were playing were actually Power Mac G5 machines hidden in the back of the enclosures. In the community these are referred to as Alphas (or Alpha XeDK systems).

A few of these boxes leaked out into the wider community and recently one of them posted how he made his G5 development system work, with lots of cool pictures. It's possible to turn one of the early 2GHz G5 machines into one of these, using a compatible graphics card (he used a flashed ATI Radeon X800 XT, but it seems easier to just use the native Mac version) and a compatible Intel network card, and a proper selection of ATA devices. There's even a term for these; they call them FrankenXenons. (A reassurance: you won't need to modify your Mac substantially to run this, and you don't need to flash the G5's firmware. On the other hand, this also means that an Alpha can be recreated or even faked with off-the-shelf hardware, and you probably shouldn't drop a lot of coin trying to get a "genuine" one unless you can verify its provenance.)

Note that this probably won't make your Mac into an Xbox 360, or at least not sufficiently to replace a proper console. Many things don't run, particularly later games, and even though it has an original Xbox emulator some of those games still glitch. But at least some early games will run and the recovery image includes a few demo applications to play with.

Eventually the Alphas were replaced with actual pre-production console hardware, called Beta, which have a strange history as Microsoft actually rounded them up, intentionally fouled them to prevent reuse, and shipped them overseas for destruction. Naturally some folks saved a few, as is what usually happens when someone realizes a prototype with potential future value is in their possession, but they have a reputation for being difficult machines.

Even as an Xbox 360 fan, I don't feel highly compelled to seek out the proper hardware to turn an otherwise perfectly useful G5 into a doorstop that runs a few early games rather badly. It certainly doesn't argue much in support for the power of the dark Microsoft side of the (X-)force, and my slim 360 is much more convenient. But as a historical artifact it's cool that you can do so, if you want to.

Tuesday, February 19, 2019

TenFourFox FPR13b1 available (now with WebP and AppleScript)

TenFourFox Feature Parity Release 13 beta 1 is now available (downloads, hashes, release notes). I took a different tack on this release because I still don't have good solutions for the missing JavaScript features currently affecting Citibank, Github and a few other sites, so I've chosen to push out some side projects I've been working on in order not to make this a wasted release. Those features are support for WebP images and support for AppleScript automation.

WebP images are an up-and-coming format based on the WebM VP8 codec, another way Google will consume the Web from the inside out, but they do have image size advantages and Firefox now supports them in Firefox 65. Google has two demonstration WebP galleries you can use to view some samples, and there are colour-managed examples in the Skia test suite. TenFourFox's WebP support currently can display lossy, lossless, transparent and colour-managed images, and will properly use any embedded colour profile. However, it is not currently AltiVec-accelerated (we do have some AltiVec VP8 code, so this should be possible at some point), and it does not yet support animated WebP images, which will appear blank. For this reason we don't pass an Accept: header indicating we accept WebP images like mainline Firefox and certain other browsers, though we will naturally try to display it if we get one. If you encounter issues related to WebP, you can try setting image.webp.enabled to false, but I'm planning to ship this support in FPR13 final, so it defaults to true.

The other support is for AppleScript. One of the few advantages of being at feature parity instead of source parity is that we can feel free to implement features mainline Firefox doesn't want or consider a current priority, and one long-standing request going back to the pre-Firefox days is AppleScript support. In fairness, this is hard to achieve in Firefox, and getting harder because of its cross-platform asynchronous nature. Many of the assumptions AppleScript makes about an application and its internal object model are routinely violated for performance reasons in Firefox, and Firefox is not primarily written in Objective-C, so there need to be bridges written to regular C++ and JavaScript, proxy objects designed, etc. Since there was never any agreement on how this internal plumbing should look, only some speculative work was ever completed, and Firefox to this day only supports the basic AppleScript suite and some limited automation through GUI scripting methods.

However, one thing that would certainly be handy for those non-daily drivers who might have a Power Mac sitting around doing nothing is to automate some tasks with it, like a kiosk or a display, or to assist with certain rote tasks. For that, AppleScript would certainly be the most painless way of doing so, so here is a first cut of AppleScript support for TenFourFox. Essentially I took that 8-year-old speculative patch, modified it to work with Firefox 45 and 10.4 (some of the dictionary actually comes from the dearly departed Camino, which had rich AppleScript support of its own), and greatly expanded its feature set to yield TenFourFox's AppleScript module. With FPR13 beta, open the AppleScript Script Editor.app, switch to the Event Log tab, and try this script (substitute your TenFourFox application name for TenFourFoxG5):

tell application "TenFourFoxG5"
  repeat with w in every browser window
    repeat with t in every tab of w
      repeat while (t is busy)
        delay 1
      end repeat
      get name of t
      get URL of t
    end repeat
  end repeat
end tell

This will iterate through every open browser window and every tab within that window, check an important synchronization property to make sure that the tab is not busy (being opened or being manipulated), and then report the name (title) and URL (location) of what's loaded in the Event Log. If you're an AppleScript jockey, you can well imagine what you can do with that information.

Tabs also have other useful properties, like plaintext and HTML to get the text or HTML contents respectively of a tab.

You might also want to create scripts for the Script menu (assuming you have that enabled) that act on text you have highlighted. TenFourFox can do that too:

tell application "TenFourFoxG5"
  repeat while (current tab of front browser window is busy)
    delay 1
  end repeat
  display dialog ("" & selected of current tab of front browser window)
end tell

If you put this script into the Script menu, then highlight some text and select this script. A dialogue box will appear with the text you have selected. (Similarly, paste it into the Script Editor and run it to see.)

You can also turn TenFourFox into an automated kiosk. Here's a script that opens a new window, makes it full screen, and then updates the display with what the New York state traffic cameras see on the Long Island Expressway every 15 seconds. It uses a second synchronization property called opening to determine when it's safe to manipulate the new window. (To stop it, Alt-Tab to the Script Editor, click Stop, then return to TenFourFox and close the fullscreen window with Cmd-W.)

tell application "TenFourFoxG5"
  activate
  make new browser window
  repeat while (opening)
    delay 1
  end repeat
  tell front browser window
    set fullscreen to true
    set URL to "https://511ny.org/map/Cctv/428834--20"
    repeat
      delay 15
      reload current tab
    end repeat
  end tell
end tell

TenFourFox can also be automated with GUI scripting as well, which can be used to manipulate the pulldown menus and even deliver clicks and keyboard events on web pages and the browser chrome. The complete dictionary, like any scriptable app, can be viewed from the Script Editor's Open Dictionary... option. Do note there are a few gotchas and a few things that don't work as expected, and you cannot currently control foxboxes with AppleScript even if they use FPR13; you can read all about the current state of AppleScript support and get many more examples of scripting on the TenFourFox Github AppleScript wiki entry. It's not a perfect mapping of Firefox/TenFourFox onto AppleScript, but it's much better than mainline Firefox which can barely be script-controlled at all. Please consider this support to be a work in progress and there may be more bugs and features yet to add(ress). Post your comments as usual.

Like I say, I'm still concerned over the deficiencies accumulating in the browser that I don't know what to do with and don't have an easy means to patch into the browser core. That said, keep in mind that even if we did try to get a port of 52 off the ground to address these problems -- the functionality of which wouldn't guaranteed and has several major changes which would badly compromise TenFourFox's platform base -- we'd just have different deficiencies once Fx60 becomes the typical minimum, so it only delays the inevitable, and the Rust requirement for 54+ makes any later wholesale port impossible. Nevertheless, in the meantime these new features, although admittedly incomplete, at least give some additional functionality to the browser, and that's not worth nothing.

FPR13 will go final with Firefox 66 on March 19.