Keeping Time
Time is an illusion. Lunchtime double so. (Douglas Adams.)
This chapter explains how a Linux system keeps time, and what you need to do to avoid causing trouble. Usually, you don't need to do anything about time, but it is good to understand it. The concept of localtime Time measurement is based on mostly regular natural phenomena, such as alternating light and dark periods caused by the rotation of the planet. The total time taken by two successive periods is constant, but the lengths of the light and dark period vary. One simple constant is noon. Noon is the time of the day when the Sun is at its highest position. Since (according to recent research) the Earth is round, noon happens at different times in different places. This leads to the concept of local time. Humans measure time in many units, most of which are tied to natural phenomena like noon. As long as you stay in the same place, it doesn't matter that local times differ. As soon as you need to communicate with distant places, you'll notice the need for a common time. In modern times, most of the places in the world communicate with most other places in the world, so a global standard for measuring time has been defined. This time is called universal time (UT or UTC, formerly known as Greenwich Mean Time or GMT, since it used to be local time in Greenwich, England). When people with different local times need to communicate, they can express times in universal time, so that there is no confusion about when things should happen. Each local time is called a time zone. While geography would allow all places that have noon at the same time have the same time zone, politics makes it difficult. For various reasons, many countries use daylight savings time, that is, they move their clocks to have more natural light while they work, and then move the clocks back during winter. Other countries do not do this. Those that do, do not agree when the clocks should be moved, and they change the rules from year to year. This makes time zone conversions definitely non-trivial. Time zones are best named by the location or by telling the difference between local and universal time. In the US and some other countries, the local time zones have a name and a three letter abbreviation. The abbreviations are not unique, however, and should not be used unless the country is also named. It is better to talk about the local time in, say, Helsinki, than about East European time, since not all countries in Eastern Europe follow the same rules. Linux has a time zone package that knows about all existing time zones, and that can easily be updated when the rules change. All the system administrator needs to do is to select the appropriate time zone. Also, each user can set his own time zone; this is important since many people work with computers in different countries over the Internet. When the rules for daylight savings time change in your local time zone, make sure you'll upgrade at least that part of your Linux system. Other than setting the system time zone and upgrading the time zone data files, there is little need to bother about time. The hardware and software clocks A personal computer has a battery driven hardware clock. The battery ensures that the clock will work even if the rest of the computer is without electricity. The hardware clock can be set from the BIOS setup screen or from whatever operating system is running. The Linux kernel keeps track of time independently from the hardware clock. During the boot, Linux sets its own clock to the same time as the hardware clock. After this, both clocks run independently. Linux maintains its own clock because looking at the hardware is slow and complicated. The kernel clock always shows universal time. This way, the kernel does not need to know about time zones at all. The simplicity results in higher reliability and makes it easier to update the time zone information. Each process handles time zone conversions itself (using standard tools that are part of the time zone package). The hardware clock can be in local time or in universal time. It is usually better to have it in universal time, because then you don't need to change the hardware clock when daylight savings time begins or ends (UTC does not have DST). Unfortunately, some PC operating systems, including MS-DOS, Windows, and OS/2, assume the hardware clock shows local time. Linux can handle either, but if the hardware clock shows local time, then it must be modified when daylight savings time begins or ends (otherwise it wouldn't show local time). Showing and setting time In Linux, the system time zone is determined by the symbolic link /etc/localtime. This link points to a time zone data file that describes the local time zone. The time zone data files are located at either /usr/lib/zoneinfo or /usr/share/zoneinfo depending on what distribution of Linux you use. For example, on a SuSE system located in New Jersey the /etc/localtime link would point to /usr/share/zoneinfo/US/Eastern. On a Debian system the /etc/localtime link would point to /usr/lib/zoneinfo/US/Eastern. If you fail to find the zoneinfo directory in either the /usr/lib or /usr/share directories, either do a find /usr -print | grep zoneinfo or consult your distribution's documentation. What happens when you have a users located in a different timezone? A user can change his private time zone by setting the TZ environment variable. If it is unset, the system time zone is assumed. The syntax of the TZ variable is described in the tzset manual page. The date command shows the current date and time. For example: $ date Sun Jul 14 21:53:41 EET DST 1996 $ That time is Sunday, 14th of July, 1996, at about ten before ten at the evening, in the time zone called ``EET DST'' (which might be East European Daylight Savings Time). date can also show the universal time: $ date -u Sun Jul 14 18:53:42 UTC 1996 $ date is also used to set the kernel's software clock: # date 07142157 Sun Jul 14 21:57:00 EET DST 1996 # date Sun Jul 14 21:57:02 EET DST 1996 # See the date manual page for more details; the syntax is a bit arcane. Only root can set the time. While each user can have his own time zone, the clock is the same for everyone. Beware of the time command. This is not used to get the system time. Instead it's used to time how long something takes. Refer to the time man page. date only shows or sets the software clock. The clock commands synchronizes the hardware and software clocks. It is used when the system boots, to read the hardware clock and set the software clock. If you need to set both clocks, you first set the software clock with date, and then the hardware clock with clock -w. The option to clock tells it that the hardware clock is in universal time. You must use the option correctly. If you don't, your computer will be quite confused about what the time is. The clocks should be changed with care. Many parts of a Unix system require the clocks to work correctly. For example, the cron daemon runs commands periodically. If you change the clock, it can be confused of whether it needs to run the commands or not. On one early Unix system, someone set the clock twenty years into the future, and cron wanted to run all the periodic commands for twenty years all at once. Current versions of cron can handle this correctly, but you should still be careful. Big jumps or backward jumps are more dangerous than smaller or forward ones. When the clock is wrong The Linux software clock is not always accurate. It is kept running by a periodic timer interrupt generated by PC hardware. If the system has too many processes running, it may take too long to service the timer interrupt, and the software clock starts slipping behind. The hardware clock runs independently and is usually more accurate. If you boot your computer often (as is the case for most systems that aren't servers), it will usually keep fairly accurate time. If you need to adjust the hardware clock, it is usually simplest to reboot, go into the BIOS setup screen, and do it from there. This avoids all trouble that changing system time might cause. If doing it via BIOS is not an option, set the new time with date and clock (in that order), but be prepared to reboot, if some part of the system starts acting funny. Another method would be to use either hwclock -w or hwclock --systohc to sync the hardware clock to the software clock. If you want to sync your software clock to your hardware clock then you would use hwclock -s or hwclock --hwtosys. For more information on this command read man hwclock. NTP - Network Time Protocol A networked computer (even if just over a modem) can check its own clock automatically by comparing it to the time on another computer known to keep accurate time. Network Time Protocol (or NTP) does exactly that. It is a method of verifying and correcting your computer's time by synchronizing it with a another system. With NTP your system's time can be maintained to within milliseconds of Coordinated Universal Time. Visit http://www.time.gov/about.html for more info. For more casual Linux users, this is just a nice luxury. At my home all our clocks are set based upon what my Linux system says the time is. For larger organizations this "luxury" can become essential. Being able to search log files for events based upon time can make life a lot easier and take a lot of the "guess work" out of debugging. Another example of how important NTP can be is with a SAN. Some SAN's require NTP be configured and running properly to allow for proper synchronization over filesystem usage, and proper timestamp control. Some SANs (and some applications) can become confused when dealing with files that have timestamps that are in the future. Most Linux distributions come with a NTP package of some kind, either a .deb or .rpm package. You can use that to install NTP, or you can download the source files from http://www.ntp.org/downloads.html and compile it yourself. Either way, the basic configuration is the same. Basic NTP configuration The NTP program is configured using either the /etc/ntp.conf or /etc/xntp.conf file depending on what distribution of Linux you have. I won't go into too much detail on how to configure NTP. Instead I'll just cover the basics. An example of a basic ntp.conf file would look like: # --- GENERAL CONFIGURATION --- server aaa.bbb.ccc.ddd server 127.127.1.0 fudge 127.127.1.0 stratum 10 # Drift file. driftfile /etc/ntp/drift The most basic ntp.conf file will simply list 2 servers, one that it wishes to synchronize with, and a pseudo IP address for itself (in this case 127.127.1.0). The pseudo IP is used in case of network problems or if the remote NTP server goes down. NTP will synchronize against itself until the it can start synchronizing with the remote server again. It is recommended that you list at least 2 remote servers that you can synchronize against. One will act as a primary server and the other as a backup. You should also list a location for a drift file. Over time NTP will "learn" the system clock's error rate and automatically adjust for it. The restrict option can be used to provide better control and security over what NTP can do, and who can effect it. For example: # Prohibit general access to this service. restrict default ignore # Permit systems on this network to synchronize with this # time service. But not modify our time. restrict aaa.bbb.ccc.ddd nomodify # Allow the following unrestricted access to ntpd restrict aaa.bbb.ccc.ddd restrict 127.0.0.1 It is advised that you wait until you have NTP working properly before adding the restrict option. You can accidental restrict yourself from synchronizing and waste time debugging why. NTP slowly corrects your systems time. Be patient! A simple test is to change your system clock by 10 minutes before you go to bed and then check it when you get up. The time should be correct. Many people get the idea that instead of running the NTP daemon, they should just setup a cron job job to periodically run the ntpdate command. There are 2 main disadvantages of using this method. The first is that ntpdate does a "brute force" method of changing the time. So if your computer's time is off my 5 minutes, it immediately corrects it. In some environments, this can cause problems if time drastically changes. For example, if you are using time sensitive security software, you can inadvertently kill someones access. The NTP daemon slowly changes the time to avoid causing this kind of disruption. The other reason is that the NTP daemon can be configured to try to learn your systems time drift and then automatically adjust for it. NTP Toolkit There are a number of utilities available to check if NTP is doing it's job. The ntpq -p command will print out your system's current time status. # ntpq -p remote refid st t when poll reach delay offset jitter ============================================================================== *cudns.cit.corne ntp0.usno.navy. 2 u 832 1024 377 43.208 0.361 2.646 LOCAL(0) LOCAL(0) 10 l 13 64 377 0.000 0.000 0.008 The ntpdc -c loopinfo will display how far off the system time is in seconds, based upon the last time the remote server was contacted. # ntpdc -c loopinfo offset: -0.004479 s frequency: 133.625 ppm poll adjust: 30 watchdog timer: 404 s ntpdc -c kerninfo will display the current remaining correction. # ntpdc -c kerninfo pll offset: -0.003917 s pll frequency: 133.625 ppm maximum error: 0.391414 s estimated error: 0.003676 s status: 0001 pll pll time constant: 6 precision: 1e-06 s frequency tolerance: 512 ppm pps frequency: 0.000 ppm pps stability: 512.000 ppm pps jitter: 0.0002 s calibration interval: 4 s calibration cycles: 0 jitter exceeded: 0 stability exceeded: 0 calibration errors: 0 A slightly more different version of ntpdc -c kerninfo is ntptime # ntptime ntp_gettime() returns code 0 (OK) time c35e2cc7.879ba000 Thu, Nov 13 2003 11:16:07.529, (.529718), maximum error 425206 us, estimated error 3676 us ntp_adjtime() returns code 0 (OK) modes 0x0 (), offset -3854.000 us, frequency 133.625 ppm, interval 4 s, maximum error 425206 us, estimated error 3676 us, status 0x1 (PLL), time constant 6, precision 1.000 us, tolerance 512 ppm, pps frequency 0.000 ppm, stability 512.000 ppm, jitter 200.000 us, intervals 0, jitter exceeded 0, stability exceeded 0, errors 0. Yet another way to see how well NTP is working is with the ntpdate -d command. This will contact an NTP server and determine the time difference but not change your system's time. # ntpdate -d 132.236.56.250 13 Nov 14:43:17 ntpdate[29631]: ntpdate 4.1.1c-rc1@1.836 Thu Feb 13 12:17:20 EST 2003 (1) transmit(132.236.56.250) receive(132.236.56.250) transmit(132.236.56.250) receive(132.236.56.250) transmit(132.236.56.250) receive(132.236.56.250) transmit(132.236.56.250) receive(132.236.56.250) transmit(132.236.56.250) server 132.236.56.250, port 123 stratum 2, precision -17, leap 00, trust 000 refid [192.5.41.209], delay 0.06372, dispersion 0.00044 transmitted 4, in filter 4 reference time: c35e5998.4a46cfc8 Thu, Nov 13 2003 14:27:20.290 originate timestamp: c35e5d55.d69a6f82 Thu, Nov 13 2003 14:43:17.838 transmit timestamp: c35e5d55.d16fc9bc Thu, Nov 13 2003 14:43:17.818 filter delay: 0.06522 0.06372 0.06442 0.06442 0.00000 0.00000 0.00000 0.00000 filter offset: 0.000036 0.001020 0.000527 0.000684 0.000000 0.000000 0.000000 0.000000 delay 0.06372, dispersion 0.00044 offset 0.001020 13 Nov 14:43:17 ntpdate[29631]: adjust time server 132.236.56.250 offset 0.001020 sec If you want actually watch the system synchronize you can use ntptrace. # ntptrace 132.236.56.250 cudns.cit.cornell.edu: stratum 2, offset -0.003278, synch distance 0.02779 truetime.ntp.com: stratum 1, offset -0.014363, synch distance 0.00000, refid 'ACTS' If you need your system time synchronized immediately you can use the ntpdate remote-servername to force a synchronization. No waiting! # ntpdate 132.236.56.250 13 Nov 14:56:28 ntpdate[29676]: adjust time server 132.236.56.250 offset -0.003151 sec Some known NTP servers A list of public NTP servers can be obtained from: http://www.eecis.udel.edu/~mills/ntp/servers.html. Please read the usage information on the page prior so using a server. Not all servers have the available bandwidth to allow a large number of systems synchronizing against them. Therefore it is a good idea to contact a system's administrator prior to using his/her server for NTP services. NTP Links More detailed information on NTP can be obtained from the NTP homepage:http://www.ntp.org. Or from http://www.ntp.org/ntpfaq/NTP-a-faq.htm