Saturday, October 8, 2016

A Saturday mystery, or, locatedb considered harmful to old Macs

I've been waist-deep on AltiVec intrinsics for the last week converting some of those big inverse discrete cosine and Hadamard transforms for TenFourFox's vectorized PowerPC VP9 codec. The little ones cause a noticeable but minor improvement, but when I got the first large transform done there was a big jump in performance on this quad G5. Note that the G5, even though its vector unit is based on the 7400 and therefore weaker than the 7450's, likes long strings of sequential code it can reorder, which is essentially what that huge clot of vector intrinsics is, so I have not yet determined if I've just optimized it well for the G5 or it's generalizeable to the G4 too. My theory is that even though the improvement ratio is about the same (somewhere between 4:1 and 8:1 depending on how much data they swallow per cycle), these huge vectorized inverse transforms accelerate code that takes a lot of CPU time ordinarily, so it's a bigger absolute reduction. I'm going to work on a couple more this weekend and see if I can get even more money out of it. 720p playback is still out of the question even with the Quad at full tilt, but 360p windowed is pretty smooth and even 360p fullscreen (upscaled to 1080p) and 480p windowed can keep up, and it buffers a lot quicker.

The other thing I did was to eliminate some inefficiencies in the CoreGraphics glue we use for rendering pretty much everything (there is no Skia support on 10.4) except the residual Cairo backend that handles printing. In particular, I overhauled our blend and composite mode code so that it avoids a function call on every draw operation. This is a marginal speedup but it makes some types of complex animation much smoother.

Overall I'm pretty happy with this and no one has reported any issues with the little-endian typed array switchover, so I'll make a second beta release sometime next week hopefully. MSE will still be off by default in that build but unless I hear different or some critical showstopper crops up it will be switched on for the final public release.

When I sat down at my G5 this warm Southern California Saturday morning, however, I noticed that MenuMeters (a great tool to have if you don't already) showed the Quad was already rather occupied. This wasn't a new thing; I'd seen what I assumed was a stuck cron job or something for the last several Saturday mornings and killed it in the Activity Monitor. But this was the sixth week in a row it had happened and it looked like it had been running for over three hours wasting CPU time, so enough was enough.

The offending process was something running /usr/bin/find to find, well, everything (that wasn't in /tmp or /usr/tmp), essentially iterating over the whole damn filesystem. A couple of ps -wwjp (What Would Jesus Post?) later showed it was being kicked off as part of the update system for an old Unix dragon of yore, locate.

There are no less than three possible ways to find files from the command line in OS X macOS. One is the venerable find command, which is the slowest of the lot (it uses no cache) and the predicates can be somewhat confusing to novices, but is guaranteed to be up-to-date because it doesn't rely on a pre-existing database and will find nearly anything. The second is of course Spotlight, which is accessible from the Terminal using the mdfind command. There are man pages for both.

The third way is locate, which is easier than find and faster because it uses a database for quick lookups, but less comprehensive than Spotlight/mdfind because it only looks for filenames instead of within file content as well, and the updater has to run periodically to stay current. (There's a man page for it too.) It would seem that Spotlight could completely supersede locate, and Apple thinks so too, because it was turned into a launchd .plist in 10.6 (look at /System/Library/LaunchDaemons/com.apple.locate.plist) and disabled by default. That's not the case for 10.5 and previous, however, and I have so many files on my G5 by now that the runtime to update the locate database is now close to five hours -- on an SSD! And that's why it was still running when I sat down to do work after breakfast.

I don't use locate because Spotlight is more convenient and updates practically on demand instead of weekly. If you don't either, then niced or not it's wasted effort and you should disable it from running within your Mac's periodic weekly maintenance tasks. (Note: on 10.3 and earlier, since you don't have Spotlight, you may not want to do this unless locate's update process is tying up your machine also.) Here's how:

  • On 10.5, the weekly periodic script can be told specifically not to run locate.updatedb. Edit /etc/defaults/periodic.conf as root (such as sudo vi /etc/defaults/periodic.conf -- you did fix the sudo bug, right?) and set weekly_locate_enable to "NO".

  • On 10.4 and before (I checked this on my 10.2.8 strawberry iMac G3 as well, so I'm sure 10.3 is the same), the weekly script doesn't offer this option. However, it does check to see if locate.updatedb is executable before it runs it, so simply make it non-executable: sudo chmod -x /usr/libexec/locate.updatedb

Now for some 8-Bit Weapon ambient (de)programming with a much more sedate G5 into the rest of the weekend.

6 comments:

  1. Shouldn't it be
    sudo vi /etc/defaults/periodic.conf
    instead of
    sudo vi /etc/periodic/defaults
    on 10.5?

    ReplyDelete
  2. MenuMeters huh? Have you ever herd of iStat Pro? It was this cool free Dashboard widget that could show you everything right down to the network interface activity. Unfortunately they abandoned it in favor of the newer iStat Menus (essentially the same thing as MenuMeters but proprietary) so although it can still be found on the internet out there, you'll have to do some digging.

    ReplyDelete
    Replies
    1. I have, but I don't like Dashboard, and I liked how MenuMeters goes right in the menubar.

      Delete
    2. MenuMeters is the handiest application I've found for worrying unnecessarily about my RAM usage :P
      I've got it running on all my Macs with an OS X new enough to support it.

      Delete

Due to an increased frequency of spam, comments are now subject to moderation.