As always, barring major issues, the browser will be released officially late Monday Pacific time, along with unveiling the new El Spoofistan design for the main page.
In other ecosystem news, I am incredibly delighted to see Tenfourbird 38.3 from our anonymous builder in the land of the Rising Sun, and it works great -- I use it for (appropriately enough) reading news.mozilla.org. Try it. You'll like it.
Yes, I confess I actually do own two Intel Macs, an old 2007 Core 2 Duo Mac mini running 10.6 which mostly serves as a test machine, and a 2014 i7 MacBook Air which I use for taxes and Master's program homework. The MBA reminds me regularly of why I preferred the New World Power Mac days, and why my daily drivers are still all Tiger PowerPC. Lately every year when Apple issues their annual update I get a bit nervous because their quality assurance seems to have gone right down the pot -- I think the beta testers just twiddle a couple buttons and call it good for golden master, and never mind when everyone's machines explode because they might actually have customized it a bit, or something similarly empowering that Apple doesn't want you doing with your overpriced appliances. (More on that when we get to my overall gripe at the end.)
Part of what makes my trepidation more acute is that I actually do write software that can run on a current Intel Mac from time to time, despite my reputation as a Power Mac-clinging troglodyte. Now, this software is still truly Universal in the strictest sense of the term -- I build on my G5 against the 10.4 universal SDK to make applications that generally run on any Mac OS from 10.4 till now, on any Power Mac and on any Intel Mac, and I even found a version of SDL 1.2 (1.2.14) that happily runs in the same environments on all systems without tripping any deprecation warnings so far. I'll be talking about one particular app in the very near future because not only is it Universal ppc/i386, it also includes AltiVec support, so it's actually 750/7400/i386. Keep an eye out for that column. Fortunately El Capitan didn't break the ones I write based on that environment.
Next, I turned to GopherVR and Mosaic-CK, my rebuilds of two venerable Motif-based X11 applications (the GopherVR client, allowing you to view gopherspace graphically, and an updated port of NCSA Mosaic that doesn't immediately barf on newer pages). These use a special launcher program to install a Dock icon and transfer control to the binary under X11. They both crashed immediately. I looked in the console and found that they were unable to run OpenMotif, which they should have detected, but I said no problem and got out my installer of IST OpenMotif 10.5+ which worked perfectly in Yosemite. On El Cap, it wouldn't install.
Why? Blame the new System Integrity Protection, which amongst other things blocks write access to certain directories, in this case /usr, even if you're root ("rootless protection"). OpenMotif expects to install itself to (more or less) /usr/OpenMotif. El Crap won't let it.
Now, you can keep your superior security snark out of my comments, thanks. I get why SIP exists, because users are stupid, and SIP saves them (somewhat) from themselves. At least so far, unless some gaping kernel hole is discovered, it looks pretty hard to toast an SIP-locked installation other than via hardware failure, and yes, if you're willing to jump through a couple hoops, you can turn it off. But you don't have to be particularly clairvoyant to realize Apple won't let you do that forever, and even now it's mostly all or nothing unless you turn SIP off and tinker with its configuration file. Frankly, making users go to all that trouble isn't a good way to distribute software.
So the first thing I did was patch OpenMotif to run from /Applications, which isn't protected (yes, /Library might be more appropriate, but I had to patch a couple paths embedded in the libraries in place and the length matched better), by changing all the linkages to the new path with a Perl script I dashed off and doing a couple direct binary changes. It looked sane, so I put it in /Applications/OpenMotif21 and rebuilt the apps on the G5 to link against that. It worked fine on 10.4 and 10.6, but on the 10.11 MBA they still crashed.
This time, the OpenMotif libraries could be found, but they were linking against /usr/X11R6, because that's where the universal X11 libraries are in the 10.4 universal SDK and every version of OS X from 10.5 to 10.10 had a symlink to /usr/X11 so it all just worked. Guess what new version of OS X doesn't? And guess what version prevents you from modifying /usr to add that link because of SIP?
I toyed with a couple solutions, but the simplest was to lipo a second i386 binary from the main one (since all Power Macs will run the original binary pointing to /usr/X11R6 fine) at the time the app package is built and rewrite all its linkages to /usr/X11 instead. Then, when the launcher starts, it figures out which binary to run depending if /usr/X11R6 exists or not, and transfers control to that. It's ugly and it bloats the app by about 25%, but it's transparent to the user, at least, and it doesn't require the user to have developer tools installed or I'd just have the launcher do the stripping and rewriting on the fly.
After that, I fixed a couple more bugs (including the original one where I had a short-circuit in the prerequisites detector) and packed everything up for distribution, and now you can use GopherVR and Mosaic-CK once again. Make sure you have X11 or XQuartz installed first, and then grab my patched OpenMotif from SourceForge (choose the 10.4 package for Tiger, or the 10.5 package for Leopard through El Capitan). Just drag the folder to /Applications without changing the name or any of the contents, and then run either GopherVR and Mosaic-CK's launcher app, and it should "just work" on any 10.4+ Mac just as it did before.
Why does SIP annoy me most, though, aside from making my binaries more complex and my headache larger? Simply put, I don't like the feeling I don't own my computer, and I'm getting that feeling more and more from Apple. I feel this much less on my Tiger boxes because I can patch them up manually and improve their security and functionality, or alter the way the OS is laid out to suit my taste and needs and how things are installed and activated, and I'm quite sure Apple has great concerns about allowing that on what they consider to be an "appliance." In fact, the irrepressible cynic in me suspects part of SIP's purpose is not just security -- it also has the (to Apple) desirable side effect of forcing most systems to exist in a specific uniform state so that installations and upgrades are more deterministic instead of allowing a (dangerously?) clueful user to muck about at will. While predictions that Gatekeeper would become locked in stone and unsigned apps would be never be allowed to run even by request have not yet come to pass, a lot fewer people will be inconvenienced by SIP than by Gatekeeper except for nutbag tinkerers and hackers like me, and Apple has little downside to making it permanent in a future version of OS X. That means one day you may not be able to change the OS at all except through those changes Apple authorizes, and that would really suck. It would also drive me to Linux on commodity hardware, because if system limitations mean I can't find a way to run my custom apps on a current Mac that run just fine on my G5 daily driver, then why have a current Mac? As it turns out, I'm not the only one thinking about that. What's Apple going to break next year, my legs?
Geez, Tim.
(*apologies to TMBG, "Dinner Bell" from Apollo 18)
Anyway, ObTenFourFox news. The two biggest crash bugs are hopefully repaired, i.e., issues 308 and 309. Issue 308 turned out to be related to a known problem with our native PowerPC irregexp implementation where pretty much anything in the irregexp native macroassembler that makes a OS X ABI-compliant function call goes haywire, craps on stack frames, etc. for reasons I have been unable to determine (our Ion ABI to PowerOpen ABI thunk works fine everywhere else). In this situation, the affected sites uncovered an edge case with checking for backreferences with UTF-16 text that (surprise) makes an ABI-compliant call to an internal irregexp function and stomped over rooted objects on the stack. I've just rewritten it all by hand the same way I did for growing the backtrack stack and that works, and that should be the last such manifestation of this problem. Assuming it sticks, the partial mitigation for this problem that I put in 38.3 will no longer be necessary and will be pulled in 38.5 since it has a memory impact.
Issue 309 was a weird one. This was another Apple Type Services-barfing-on-a-webfont bug that at first blush looked like we just needed to update the font blacklist (issue 261), except it would reliably crash even 10.4 systems which are generally immune. Turns out that null font table blobs were getting into Harfbuzz (our exclusive font shaper) from ATS that we weren't able to detect until it was too late, so I just wallpapered a bit in Harfbuzz to handle the edge case and then added the fonts to the blocklist as well. This doesn't affect Firefox, which has used CoreText exclusively ever since they went 10.6+. Both fixes will be in 38.4.0.
The problem with restoring from LZ4-compressed bookmark backups remains (issue 307) and I still don't know what's wrong. It doesn't appear to be an endian problem, at least, which would have been relatively easy to correct. The workaround is to simply not compress the backups (i.e., roll back to 31) until I figure out why it isn't functioning correctly and it will be in 38.4.0 also. I've also been doing some work on seeking within our MP3 audio implementation, which is the last major hurdle before enabling it by default. It won't be enabled in 38.4.0, but there might be enough of it working at the time of release for you to test it before I publicly roll it out (hopefully 38.5).
Also, some of you might have been surprised that I didn't post anything at the time about Mozilla's new plan to pull plugins (TenFourFox has of course been plugin-free since 6.0). I didn't comment on it frankly because I always figured it was inevitable, for many of the same reasons I've laid out before ad nauseam. Use SandboxSafari if you really need them.
Anyway, moving on to today's post, one of my hobbies is weird cameras. For example, I use a Fujifilm Real 3D W3 camera for 3-D images, which is a crummy 2D camera but takes incredible 3D pictures like nothing else. The camera takes both stills and video that I can display on my 3D HDTV, or I can use a custom tool (which I wrote, natch) to break apart its MPO images into right and left JPEGs and merge them into "conventional" anaglyphs with RedGreen (from the author of iCab, as it happens). I haven't figured out yet how to decode its AVI video into L/R channels, but I'm working on it. Post in the comments if you know how.
Panoramas have also been a longtime interest of mine, facilitated by QuickTime VR, another great software technology Apple completely forgot about (QTVR works fine through 10.5 but 10.6+ QuickTime X dropped it as a "legacy" format; you have to install QuickTime 7 to restore support). Most mobile devices still do a pretty crappy job on panos, and even though iOS and Android's respective panorama modes have definitely improved, they could scarcely have gotten worse. My original panorama workflow was to take one of my Nikon cameras and put it on a tripod, march out angles, and labouriously stitch them together with Hugin or QTVR Authoring Studio. QTVR Authoring Studio, by the way, works perfectly in Classic under 10.4, yet another reason I remain on Tiger forever. Unfortunately this process was not fun to shoot or edit, required much fiddling with exposure settings if lighting conditions were variable, usually had some bad merge areas that required many painstaking hours with an airbrush, and generally yielded an up-down field of view as roomy as an overgrown mail slot even though the image quality was quite good.
The best way to do panoramas is to get every single angle at once with a catadioptric camera. These can be added as external optics -- unfortunately with varying levels of compromises -- to an existing camera, or you can do substantially better by getting a camera expressly built for that purpose, which in this case is the Ricoh Theta m15.
The Ricoh Theta cameras are two fisheye lenses glommed together for a 360 degree view in both axes generated as an equirectangular image; the newest member, the Theta S, just came out (but too late for my trip). The m15 comes in four colours, all of them silly, but mine is blue. You can either take free shots with the button, or you can control it with a smartphone (iOS and Android apps available) over Wi-Fi using either the official Ricoh app or the free Android HDR one (tripod strongly advised).
The m15 doesn't take exceptional images in low light, and the resolution is a bit low (6.4 megapixels at 3584x1792, but remember that it's two images that the camera firmware glues together, and there's an awful lot of spherical aberration due to the design). But it does work, and you shouldn't be scared by the reviews and instructions saying you need a current Intel Mac or Windows PC to view your images. That's a damn lie, of course -- connect the camera over USB and Image Capture will happily download them (even 10.4), or have the smartphone apps download the images to your phone and send them over Bluetooth or E-mail them. Either way, the images work perfectly as QTVR panos; you don't need the Ricoh desktop software. When you load the image into QTVR Authoring Studio and make sure it's oriented horizontally, it "just works" with no tweaking necessary:
And here's a frame from the result, in QuickTime 7 (rendered out at 1024x768, high quality, 100% Photo JPEG compression):
Unlike the ugly map-on-a-sphere distortion the Ricoh apps cause, the QTVR pano looks just like "you're there," and you can share it with all your friends without any other software other than QuickTime. You can look all the way up and down, ignoring the camera's limp effort to edit itself and its base out, and there are no stitch lines or agonizing hours of retouching. With just a few seconds of processing, I'm back along the side of Interstate H3, looking over the Kaneohe bay once again. A perfect memory, perfectly captured. Isn't that what you bought the camera for in the first place?
The new Theta S bumps the resolution to 14.4 megapixels (5376x2688), and both the m15 and S are capable of video with the S offering 1080p quality. In fact, I'm so pleased with even the lower resolution of the m15 that I'll be picking up an S very soon, and my suspicion is it will work just as well. It's wonderful to see that an old tool like QTVR Authoring Studio still works flawlessly with current cameras, and given that QTVR-AS was never written for OS X, it's another example of how Classic is the best reason to still own an Power Mac.
When I started getting alarms that oslo was unresponsive from the systems that couldn't dump backups on it anymore, I tried to bring it back up after work and noticed I was missing the array's core HFS+ volume (the array is partitioned into three). While the array's UFS volume seemed intact, none of the files verified; worse, not only was the HFS+ volume on it completely hosed, DiskWarrior actually crashed trying to repair it after throwing an error I'd never seen before (2351, -36 in case anyone is searching in the future).
Now, this is a backup array, so it could be dispensed with, but I wanted to understand what was going on first before wiping it. In the interest of ruling out the controller, I first decided to see if it was a problem with the array hardware and connected the RAID over FireWire to the iBook instead. To my surprise and delight, the files on the UFS volume checked out! I quickly booted DiskWarrior on the iBook and managed to get the HFS+ volume repaired -- it was pretty hideously mangled but spot checks on the files seemed to validate. So that ruled out the array.
At this point I concluded it must have been something wrong with the FW800 card. I'm notorious for keeping large amounts of spares on hand, so I got a spare FW800 card out that I'd bought as a Fry's special six years ago, pulled the old one from the bottom slot and installed the new one in the same place, and connected the array. This time, no volumes mounted, and System Profiler actually crashed when I asked it to enumerate FireWire devices. I powered off the system and said something intemperate, then started to wonder if it was something about the slot or (grr) the logic board. Fortunately, I could dispense with the Grappler+ since it was just occupying space, so I pulled that and put the new card in its slot instead. Everything mounted. Whew! Conclusion: bad PCI slot, possibly bad card as well, not clear if or which one caused the other, but I'm not going to take chances -- that card's going in the eWaste bin. Either way, the moral of this story is to not only keep backups, but keep spare parts on hand, because you never know when you'll need them. And oslo has a complete body double in the stock closet for the day it might blow its logic board entirely. Still, not bad for a machine that was built 15 years ago.
Now to the main event. I mentioned a while back a couple of secret projects I've been working on, and while one of them is probably going to get invalidated by Google again in the very near future, this second one has bigger import: no less than a safer, sandboxed way to run Flash and other plugins on Tiger. Let me introduce you to SandboxSafari.
First, let's be painfully clear about two important points: both Flash and Java remain unsafe to run on Power Macs, and TenFourFox's no-plugins policy remains inviolate. That's not ever going to change. But there are still sites people want to visit that insist on requiring a plugin and some of the sites either still work or can be coerced to work with the older version of Flash Player available for PowerPC. While not all the problems can be mitigated, it seemed to me that there could be a safer way of doing so that would reduce a potential attack profile.
Because our primary operating system of interest is Tiger (Leopard is supported, but I've always been frank that the best reason to still own a Power Mac is to run Classic apps, and that means 10.4), we can't use the Leopard sandbox, and the Leopard sandbox has at least one sandbox escape that was never fixed (though I suppose in the future it could be part of a blended implementation). So SandboxSafari takes a different approach: it runs a very limited WebKit instance in a separate process as nobody so that it doesn't run within TenFourFox or even run as you. By limiting the functionality it exposes and the privileges it can wield, it reduces the chance of an unsafe operation because it can't do many such operations.
In fact, SandboxSafari is so limited you can't even use it as a regular browser; there's no tabs, no downloads, only a single window and almost no chrome except for a right-click context menu for navigation. You feed it URLs through Launch Services, not through a typical address bar -- because there isn't one! It can't even save its own settings, nor can any plugin it instantiates, let alone anything else. The overriding idea is "as little as possible to go wrong."
So how do you use it? TenFourFox integrates with SandboxSafari through an enabler add-on included in the package. Let's say you're on one of those troublesome sites and you need Flash to operate on it. Just right-click on an open area of the page and from the context menu select the option to pass the URL to SandboxSafari (or pass it and close the tab simultaneously, such as a video site you don't need to keep open in TenFourFox). SandboxSafari will open to that URL; when you're done, just close the window and it will return to the previous app (invariably TenFourFox). You can drive SandboxSafari from your own application with AppleEvents, btw; see the openurl tool in the source code bundle.
SandboxSafari is not at all complete protection. Even though crashes are limited to the SandboxSafari process, it may still be possible for a malicious or misbehaving site to trigger other OS bugs, and while it's designed to resist modifying old files as well as creating new files, it may still be able to read other files and possibly be manipulated to upload them. A subverted plugin may also be able to activate input contexts that capture all keyboard events or draw things that look like password dialogues, and while these would disappear when SandboxSafari quits, they could potentially grab data anyway if you type in sensitive information while SandboxSafari is running. These kinds of operations are not generally dependent on the user ID in use or cannot otherwise be blocked with this method of sequestration. There are other limitations with what it can access and you should read the documentation thoroughly beforehand so you understand why.
SandboxSafari is also provided to you "best effort" and strictly "as-is." It's a means to get obsolete, insecure software to run a little more safely, but the software in question is unmaintained and known to have problems, and that means you should expect to have problems with it too. Because I don't want to turn my life into a living hell with people who can't read and don't care sending me complaints I don't want, the policy is explicit: bug reports that do not include fixes will be ignored (unless it's an obvious security issue over and above the security issues I already know it has). Don't write it here, don't post it to Tenderapp, don't E-mail me. If you're having trouble with it and you don't know why or don't care to find out how to fix it, then don't use it. I'm serious. I have enough to worry about with TenFourFox without regretting anything else I maintain. :P
Download SandboxSafari here, after reading the page thoroughly, and God have mercy on your souls.
Try as I might, however, I could not find any XUL in it (there is no XUL, only WebExtensions).
TenFourFox 38.3.0 is now available for testing (release notes, hashes, downloads). Currently I'm tracking two confirmed bugs and one questionable one, none of which are fully fixed yet. We have an issue with what appears to be exact rooting and may be partially a compiler bug, but I can only reproduce it on Github, and only specifically with one particular operation. The patch I added to 38.3.0 seems to wallpaper the intermittent crash that can result from it though not the incorrect behaviour it causes with the JITs (but it does restore correct behaviour with the naked interpreter, which was not the case in 38.2.x), although I don't think this is the cause of some of the unreproducible crashes a few people have reported -- YMMV. In addition, the new jsonlz4 compressed JSON bookmark backup format seems to be bogus despite setting all the proper endian options near as I can tell, so I'm not sure why it's writing invalid data. If I can't figure this out, forcing it to save uncompressed backups a la TenFourFox 31 should not be exceptionally hard. The remaining questionable bug allegedly has to do with modal drop-down-sheet requesters becoming unresponsive, which in turn makes the browser halt (more accurately it goes into a loop where it ignores other events), but I can't get it to do so reliably. Because of these issues, official MP3 support is still delayed for the time being.
Disabling incremental garbage collection in lieu of periodic full collections of the tenured heap appears to be a mixed bag with respect to performance. Although it was a win on my systems, a few of you reported it made them somewhat worse, and not enough people reported a difference to really outweigh them. You can still consider trying it on your own Power Mac and seeing if that helps, but I've abandoned this line of enhancement for the time being and it remains enabled by default in 38.3. As always, the browser goes live on Monday evening Pacific time.
I've been slack on some other fun posts and I'll try to catch up on those soon as my situation permits.
Don't forget to test 38.2.1 with incremental GC disabled. See the previous post. Enjoy our new sexy wiki, too. Sexy. Yes.
