Saturday, December 20, 2014

Time, time, time, see what's become of ntpd

UPDATE: After digging through codepaths in ntpd, I'm concluding that an outbound connection -- i.e., you connecting to an external time source -- is probably not vulnerable, at least not to the most serious of the flaws below (ctl_putdata() and configure()). By default OS X does not allow inbound connections, which definitely are vulnerable. If you have changed this configuration, however, you should still update, and the sixth flaw in receive() may still apply, although the exploitability of this particular flaw is believed to be highly unlikely. The tools are still available if you want them.

The sec-buzz this time is a package of six, count 'em, six vulnerabilities in OS X's Network Time Protocol daemon, or ntpd (and possibly its associated tools, see below). NTP is one of the oldest Internet protocols still in use, starting with NTPv1 RFCed all the way back in 1985 -- in fact, the very same David L. Mills who wrote the original NTP RFC 958 is still the one maintaining it today, almost twenty years later. NTP is an extremely accurate and somewhat complex means of time synchronization that can keep Internet-connected clock devices and computers tightly coordinated with highly accurate timesources. In theory, NTP can capture time differences as small as 233 picoseconds, or 2-32 seconds; in typical network environments, it can keep accurate time within a few tens of milliseconds (currently the internal NTP server at Floodgap has an average dispersion of about 15ms), and as low as a single millisecond or less on a local network (my NetBSD/cobalt server is showing 0.2ms).

Accurate time is very important, especially for security, and virtually every modern platform offers some manner of time synchronization; almost every Un*x or Un*x-like operating system uses ntpd, the reference implementation (don't start that OpenNTPD crap here, please), including Mac OS X. And no surprises, the version included on every release of OS X is vulnerable up to and including Yosemite which only runs ntp 4.2.6.

The immediate response is, "but wait! wait! I don't run a time server!" No, you don't (probably), but if you have "Set date & time automatically" checked in Date & Time in System Preferences, you're running ntpd. (If you did the change I recommended in an earlier post to switch to intermittent ntpdate in cron, you're not running ntpd but you are still vulnerable to one of the flaws; read on.) If the remote server sends you a specifically crafted malicious packet, your computer could be commandeered to run a program under the control of the attacking server.

In practice, this attack is going to be limited for most of us for several reasons:

  • By default the Apple implementation doesn't use cryptographically authenticated time packets (flaws one, two and three).

  • If you are connecting to a well-maintained, trusted time server (time.apple.com would be an obvious one), the chances of it going bad or packets from it being intercepted and subverted are low to very low in most circumstances. (The risk of interception and subversion is higher if you don't trust the nameserver or the network, such as unencrypted Wi-Fi in a hostile environment, where you could be directed to a malicious server or someone could try to inject packets as fast as possible to arrive before an authoritative response.)

  • If you're on a Power Mac, unless the attacker knows this and sends a PowerPC-specific sequence, the circulating attacks (to flaws four, five and six) will very probably be designed for x86 and this lowers your effective risk to near zero. At worst ntpd will just crash. (If you're on an Intel Mac, though, this doesn't help you; you're the type of computer an automated scanner would target.)

  • If you don't even try to synchronize your clock, which is apparently the case for more systems than I would have thought, you aren't vulnerable at all. But this isn't a good idea.

If you are running ntpdate from cron only, and not ntpd, ntpdate is potentially vulnerable to flaw six but none of the others (again, assuming authentication is not enabled). No one is even sure if it's exploitable in its current form, and all of the mitigations above (except the last) still apply, so your risk is even lower in that case. I am in error. ntpdate is not vulnerable to this flaw.

In my situation, I have an internal NTP server which talks to a selection of publicly available stratum-1 and stratum-2 timesources. Since it's facing out on the wild wild Internet, I went ahead and upgraded that to ntp 4.2.8, which has these flaws repaired. All of my internal systems only talk to this internal NTP server which is now protected from the problem, and the only way someone's going to subvert that is by either splicing the wires or installing malicious software on the inside -- and if that happens I've got bigger problems than bad clocks. Just in case, however, I went ahead and built and updated ntpdate on my 10.4 systems so that there's no chance.

After careful consideration, I am hesitant to indiscriminately advise everyone to update ntp on your system -- I've learned from offering a rebuild of bash just how many of you can wreck your computers from Terminal :P -- unless you must connect to a risky network from time to time. Even in that case, however, it might be better just to disable "Set date & time automatically" when you're on those types of connections instead and resynch your clock when you're back on something reasonably trustworthy.

If you really can't avoid that, or you're incredibly paranoid, then here is a build of the relevant utilities for 10.4 PowerPC (it will work on 10.5 also). I had to make a change to ntp_io.c to make it build with Xcode 2.5, which I included if you want to roll your own. It only includes the core ntp tools that have relevant vulnerabilities or dependencies (ntpd ntpdate ntpdc ntpq) and none of the SNTP stuff, which is a pretty sucky way to maintain time anyhow. You will notice there is no x86 or universal build, because the current version of the source code doesn't make it easy to build fat binaries, nor instructions, because I want you to think very carefully about why you're installing this. If you're on an x86 system with a vulnerable timekeeper and no available update for this issue (10.6, say), and you're recurrently or even constantly on a hostile network, then you've got bigger problems than this one -- this is a very small bandaid on a potentially deep wound. For that matter, though, even if you're on a Power Mac, please consider what you're trying to accomplish before you install these replacements. They can replace the current ones on the system directly, or if you're running ntpdate from cron and don't have ntpd running, then just put the new ntpdate somewhere convenient like /usr/local/bin and change your crontab to run it from there.

Stay careful.

11 comments:

  1. What about the "restrict noquery" mitigation in the original ntp.org security notice?

    ReplyDelete
    Replies
    1. At least on Tiger, this is already in /etc/ntp-restrict.conf:

      restrict default kod nomodify notrap nopeer noquery
      restrict -6 default kod nomodify notrap nopeer noquery

      # localhost is unrestricted
      restrict 127.0.0.1
      restrict -6 ::1

      Delete
  2. Does this NTP vulnerability mean that any Mac running 10.6.8 can get hacked and remote controlled even if it's not running a web server? In other words, all one has to do is simply connect to the Internet and someone can take over your machine through the NTP exploit even if you're not running any servers?

    ReplyDelete
    Replies
    1. See my update above where I went through the code, but tl;dr, I don't think so for outbound connections (i.e., you connecting to a time source -- even if the time source were commandeered the worst that could happen is they'd screw up the time). For inbound connections (i.e., someone connecting _to you_ to get the time), you are definitely vulnerable, but the default configuration in OS X is to prevent this. This hasn't changed in OS X since at least 10.4 and you'd have to edit the NTP configuration to change it yourself.

      However, if you're really concerned and you don't want to try building ntp yourself, I think the best option is simply to disable automatically syncing to a time source when you're not on a trusted network. If you're on your home network, you're using a well-known timesource like Apple's and the machine doesn't travel, the risk is probably slight. Apple's OS X update today was mostly being overcautious (and for those systems running OS X Server).

      Delete
  3. Pardon my ignorance, but how do I install the fixed build of ntpd in Leopard PPC?

    ReplyDelete
    Replies
    1. I'm being intentionally vague -- if you've altered your NTP configuration to accept inbound connections, then you probably already know (or should know) where these files go. You would know you've done this because you would have had to modify ntp configuration files in the Terminal.

      If you aren't in that category, this is a different and lower priority than the bash Shellshock issue, which pretty much hits everyone. If you're concerned and you don't already know what to do with these files, I would follow the recommendation I made for the user above.

      Delete
  4. Hi there,

    Thanks again for this after I successfully followed your bash patching procedures. In OS X Tiger 10.4.11, I have renamed ntpd, ntpdate, ntpdc and ntpq to xxxx.old, then cp'd the ntp-4.28 binaries into /usr/bin and /usr/sbin as appropriate, then set root:wheel ownership for each. I now get ntpd 4.2.8@1.3265-o from ntpd --version. I have also chmod -x'd the xxxx.old files. With ntpq -p, I get a sensible output, similar to

    remote refid st t when poll reach delay offset jitter
    ==============================================================================
    *time.apple.com 17.168.198.149 2 u 1046 512 37 58.475 9.477 3.674

    but in console.log, I get messages of the following form (xxxx is obviously not what I actually see)

    ntpdate[126]: no servers can be used, exiting
    ntpd[128]: setsockopt IPV6_MULTICAST_IF 0 for xxxxx::x%x fails: Can't assign requested address
    ntpd[128]: setsockopt IPV6_MULTICAST_IF 0 for xxxxx::xxx:xxxx:xxxx:xxxx%x fails: Can't assign requested address
    ntpd[128]: mlockall(): Function not implemented

    Do these message indicate that ntpd 4.28 is not working with OS X Tiger?

    ReplyDelete
    Replies
    1. The fact you get something reasonable from ntpq -p suggests that ntpd is fine; it wasn't able to start the IPv6 multicast support but you shouldn't need that. It would not be able to compute jitter and offset if it were unable to connect. As long as it's still running (ps auxww | grep ntpd), I think it's fine. You could try to disable INCLUDE_IPV6_MULTICAST_SUPPORT and rebuild if you wanted an evening project (you'll still need my changes for ntp_io.c to get Tiger to build it).

      Delete
  5. Thanks for the prompt and as usual helpful reply. Using ps auxww | grep ntpd gives:

    xxxxx 242 3.2 0.0 27400 448 p1 S+ 10:58PM 0:00.01 grep ntpd
    xxxx 128 0.0 0.1 29588 696 ?? Ss 9:56PM 0:00.75 ntpd -c /private/etc/ntp-restrict.conf -f /var/run/ntp.drift -p /var/run/ntpd.pid

    It appears that this issue is not OS X Tiger specific, as it is affecting OS X Snow Leopard too (Probably OS X Leopard and OS X Lion?) See https://discussions.apple.com/thread/6739506?tstart=0
    which discusses the updating to ntpd 4.28 on OS X Snow Leopard in some detail.

    I have all this to come as I also have OS X 10.5 (PPC and Intel) thru to OS X 10.7 systems to update in addition to my OS X Tiger systems!

    Thanks again.

    ReplyDelete
    Replies
    1. Interesting thread. I note that my changes to ntp_io.c were the same thing that the NTP maintainers officially fixed in http://bugs.ntp.org/show_bug.cgi?id=2697 but their way is much more elegant. Either way it will be in 4.2.8p1 but it doesn't affect this.

      The thread also suggests adding rlimit memlock 0 to /etc/ntp-restrict.conf. Although that will squelch the mlockall() message, it's not really necessary; the failure is not fatal.

      Delete
    2. BTW I did also compile 4.28 myself using Xcode v2.5 for OS X Tiger and the ntp_io.c diff file from http://www.macissues.com/2014/12/24/how-to-manually-patch-ntp-for-os-x-10-6-and-10-7/
      but had similar console.logs to those of WZZZ reported in the Apple thread (page 5), and as I was also very concerned about the message "no servers can be used, exiting." and not being certain of ntpd's proper functioning, I reverted to using your 4.28 binaries for my OS X Tiger systems as a simpler yet still secure updating approach, as we have discussed above.

      Delete

Due to an increased frequency of spam, comments are now subject to moderation.