Monday, September 2, 2019

The deformed yet thoughtful offspring of AppleScript and Greasemonkey

Ah, AppleScript. I can't be the only person who's thinking Apple plans to replace AppleScript with Swift because it's not new and sexy anymore. And it certainly has its many rough edges and Apple really hasn't done much to improve this, which are clear signs it's headed for a room-temperature feet-first exit.

But, hey! If you're using TenFourFox, you're immune to Apple's latest self-stimulatory bright ideas. And while I'm trying to make progress on TenFourFox's various deficiencies, you still have the power to make sites work the way you want thanks to TenFourFox's AppleScript-to-JavaScript "bridge." The bridge lets you run JavaScript within the page and sample or expose data back to AppleScript. With AppleScript's other great powers, like even running arbitrary shell scripts, you can connect TenFourFox to anything else on the other end with AppleScript.

Here's a trivial example. Go to any Github wiki page, like, I dunno, the one for TenFourFox's AppleScript support. If there's a link there for more wiki entries, go ahead and click on it. It doesn't work (because of issue 521). Let's fix that!

You can either cut and paste the below examples directly into Script Editor and click the Run button to run them, or you can cut and paste them into a text file and run them from the command line with osascript filename, or you can be a totally lazy git and just download them from SourceForge. Unzip them and double click the file to open them in Script Editor.

In the below examples, change TenFourFoxWhatever to the name of your TenFourFox executable (TenFourFoxG5, etc.). Every line highlighted in the same colour is a continuous line. Don't forget the double quotes!

Here's the script for Github's wiki.

tell application "TenFourFoxWhatever"
    tell current tab of front browser window to run JavaScript "
 
    // Don't run if not on github wiki.
    if (!window.location.href.match(/\\/\\/github.com\\//i) ||
        !window.location.href.match(/\\/wiki\\//)) {
        window.alert('not a Github wiki page');
        return;
    }
 
    // Display the hidden links
    let nwiki=document.getElementById('wiki-pages-box').getElementsByClassName('wiki-more-pages');
    while (nwiki.length > 0) {
        nwiki.item(0).classList.remove('wiki-more-pages');
    }
 
    // Remove the 'more pages' link (should be only one)
    let jwiki=document.getElementById('wiki-pages-box').getElementsByClassName('wiki-more-pages-link');
    if (jwiki.length > 0)
        jwiki.item(0).style.display = 'none';
 
    "
end tell

Now, have the current tab on any Github wiki page. Run the script. Poof! More links! (If you run it on a page that isn't Github, it will give you an error box.)

Most of you didn't care about that. Some of you use your Power Macs for extremely serious business like YouTube videos. I ain't judging. Let me help you get rid of the crap, starting with Weird Al's anthem to alumin(i)um foil.

With comments in the five figures from every egoist fruitbat on the interwebs with an opinion on Weird Al, that's gonna take your poor Power Mac a while to process. Plus all those suggested videos! Let's make those go away!

tell application "TenFourFoxWhatever"
    tell current tab of front browser window to run JavaScript "
 
    // Don't run if not on youtube.
    if (!window.location.href.match(/\\.youtube.com\\//i) ||
        !window.location.href.match(/\\/watch/)) {
        window.alert('not a YouTube video page');
        return;
    }
 
    // Remove secondary column and comments.
    // Wrap in try blocks in case the elements don't exist yet.
    try {
        document.getElementById('secondary').innerHTML = '';
        document.getElementById('secondary').style.display = 'none';
    } catch(e) { }
    try {
        document.getElementById('comments').innerHTML = '';
        document.getElementById('comments').style.display = 'none';
    } catch(e) { }
 
    "
end tell

This script not only makes those areas invisible, it even nukes their internal contents. This persists from video to video unless you reload the page.

As an interesting side effect, you'll notice that the video area scaled to meet the new width of the browser, but the actual video didn't. I consider this a feature rather than a bug because the browser isn't trying to enable a higher-resolution stream or scale up the video for display, so the video "just plays better." Just make sure you keep the mouse pointer out of this larger area or the browser will now have to composite the playback controls.

You can add things to a page, too, instead of just taking things away. Issue 533 has been one of our long-playing issues which has been a particular challenge because it requires a large parser update to fix. Fortunately, Webpack has been moving away from uglify and as sites upgrade their support (Citibank recently did so), this problem should start disappearing. Unfortunately UPS doesn't upgrade as diligently, so right now you can't track packages with TenFourFox from the web interface; you just get this:

Let's fix it! This script is a little longer, so you will need to download it. Here are the major guts though:

    // Attempt to extract tracking number.
    let results = window.location.href.match(
        /^https:..www.ups.com.track.loc=([a-zA-Z_]+)\\&tracknum=([a-zA-Z0-9]+)\\&/
    );
    if (!results || results.length != 3) {
        window.alert('Unable to determine UPS tracking number.');
        return;
    }
 
    // Construct payload.
    let locale = results[1];
    let tn = results[2];
    let payload = JSON.stringify({'Locale':locale,'TrackingNumber':[tn]});

A bit of snooping on UPS's site from the Network tab in Firefox 69 on my Talos II shows that it uses an internal JSON API. We can inject script to complete the request that TenFourFox can't yet make. Best of all, it will look to UPS like it's coming from inside the house the browser ... because it is. Even the cookies are passed. When we get the JSON response back, we can process that and display it:

    // For each element, display delivery date and status.
    // You can add more fields here from the JSON.
    output.innerHTML = '';
    data.trackDetails.forEach(function(pkg) {
        output.innerHTML += (pkg.trackingNumber+' '+
            pkg.packageStatus+' '+
            pkg.scheduledDeliveryDate+'<p>');
        });
    }

So we just hit Run on the script, and ...

... my package arrives tomorrow.

Some of you will remember a related concept in Classilla called stelae, which were scraps of JavaScript that would automatically execute when you browse to a site the stela covers. I chose not to implement those in precisely that fashion in TenFourFox because the check imposes its own overhead on the browser on every request, and manipulating the DOM is a lot easier (and in TenFourFox, a lot more profitable) than manipulating the actual raw HTML and CSS that a stela does (and Classilla, because of its poorer DOM support, must). Plus, by being AppleScript, you can run them from anywhere at any time (even from the command line), including the very-convenient ever-present-if-you-enable-it Script menu, and they run only when you actually run them.

The next question I bet some of you will ask is, that's all fine for you because you're a super genius™, but how do us hoi polloi know the magic JavaScript incantations to chant? I wrote these examples to give you general templates. If you want to make a portion of the page disappear, you can work with the YouTube example and have a look with TenFourFox's built-in Inspector to find the ID or class of the element to nuke. Then, getElementById('x') will find the element with id="x", or getElementsByClassName('y') will find all elements with y in their class="..." (see the Github example). Make those changes and you can basically make it work. Remove the block limiting it to certain URLs if you don't care about it. If you do it wrong, look at the Browser Console window for the actual error message from JavaScript if you get an error back.

For adding functionality, though, this requires looking at what Firefox does on a later system. On my Talos II I had the Network tab in the Inspector open and ran a query for the tracking number and was able to see what came back, and then compared it with what TenFourFox was doing to find what was missing. I then simulated the missing request. This took about 15 minutes to do, granted given that I understood what was going on, but the script will still give you a template for how to do these kinds of specialized requests. (Be careful, though, about importing data from iffy sites that could be hacked or violating the same-origin policy. The script bridge has special privileges and assumes you know what you're doing.) Or, if you need more fields than the UPS script is providing, just look at the request the AppleScript sends and study the JSON object the response passes back, then add the missing fields you want to the block above. Tinker with the formatting. Sex it up a bit. It's your browser!

One last note. You will have noticed the scripts in the screen shot (and the ones you download) look a little different. That's because they use a little magic to figure out what TenFourFox you're actually running. It looks like this:

set tenfourfox to do shell script "ps x | perl -ne '/(TenFourFox[^.]+)\.app/ && print($1) && exit 0'"
if {tenfourfox starts with "TenFourFox"} then
    tell application tenfourfox
        tell «class pCTb» of front «class BWin» to «event 104FxrJS» given «class jscp»:"

This group of commands runs a quick script through Perl to find the first TenFourFox instance running (another reason to start TenFourFox before running foxboxes). However, because we dynamically decide the application we'll send AppleEvents to (i.e., "tell-by-variable"), the Script Editor doesn't have a dictionary available, so we have to actually provide the raw classes and events the dictionary would ordinarily map to. Otherwise it is exactly identical to tell current tab of front browser window to run JavaScript " and this is actually the true underlying AppleEvent that gets sent to TenFourFox. If TenFourFox isn't actually found, then we can give you a nice error message instead of the annoying "Where is ?" window that AppleScript will give you for orphaned events. Again, if you don't want to type these scripts in, grab them here.

No, I'm not interested in porting this to mainline Firefox, but the source code is in our tree if someone else wants to. At least until Apple decides that all other scripting languages than the One True Swift Language, even AppleScript, must die.