TopHome
<2023-05-05 Fri>technetworking

Setting system time or an accidental deep-dive into NTP

So, for some reason - that reason being that NTP sync was switched off, my Linux system's time was lagging by around 5 minutes.

Firstly, the easy solution is to enable NTP and be done with it. Apparently, these days the frontend to use for this is timedatectl since NTP stuff has gone into systemd. If you want to do this, there is also info available on Indian NTP servers here.

But, you can manually set the time directly using the date command: 

$ date -s "<string>" # or
$ date -s @<unix timestamp>

But, how to get the right values for these? You know the time from a watch or a phone, but not in this format.

This stack overflow answer had a nice bit about simply lifting time of the HTTP response from a site like Google. Good idea. Though the time it returns is in GMT, setting it seems to work correctly.

But, I wanted some thing more official than depending on a website like Google.

So, I started searching for the Indian standards organization and found NPL - the National Physical Laboratory of India which comes under the CSIR - Council of Scientific and Industrial Research. The mandate of measuring and keeping standards upto date comes under the aegis of NPL.

So, you can use the "Date" response on the header from this website - assuming that they, of all people, keep their websites upto date.

But, you can go to the page they maintain for this exact purpose: https://www.nplindia.org/clockcode/html/

This is great for you as a human, but how do you use the time from this? Looking at the html source of the page, it seems the page is generated with the timestamp at call times and then there is a bunch of processing done using js to display it. 

wget -qO- https://www.nplindia.org/clockcode/html/ | grep "^timer" | awk '{print $3}' | awk -F* '{print $1}'

This command gets you the exact time as seconds since epoch that NPL India thinks is correct.

This gets us what we want, but a bit ugly.

It turns out that they run a NTP server at: "time.nplindia.org". Now, that sounds a lot more cleaner. But, how to get the time from there? We need a ntp client, but those seem difficult to get…

The simplest option I found is the Python NTP client library. 

$ pip install ntplib
$ python
>>> import ntplib
>>> c = ntplib.NTPClient()
>>> res = c.request('time.nplindia.org')
>>> res.tx_time
1683268307.4614358

It turns out that the NTP protocol is a simple one built on UDP. There are number of implementations available out there in C, for example this one: https://github.com/ForgotFun/ntpc/blob/master/ntpc.c

That made me search for even simpler version and I found this winner in Python: https://github.com/lettier/ntpclient/blob/master/source/python/ntpclient.py

Looking at these 2 pages together made me fall into the rabbit hole of trying to understand what was happening. In this, Kavya joined me and in about 30 minutes, we understood everything.

We can have a proper understanding of everything by going through the RFCs mentioned in the Wikipedia page: https://en.wikipedia.org/wiki/Network_Time_Protocol, but who has time for that? Instead, we went in heas first, decoding bits, making mistakes - but that is the best way to learn things!

Let me present what we learnt, bottom up.

NTP is based on UDP on port 123, so we need to open a UDP connection and send some packet to which we will get a response back with the timestamp (well, a lot of things actually. But, we are only interested in the simple time component). The request itself seemed to be pretty light looking at the 2 implementations above - so the aim was to craft and send a NTP request and use Wireshark to decode the response.

NTP packets look like the following: 

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9  0 1
+---+-----+-----+---------------+---------------+----------------+
|LI | VN  |Mode |    Stratum    |     Poll      |   Precision    | 0
+---+-----+-----+---------------+---------------+----------------+
|                          Root  Delay                           | 1
+----------------------------------------------------------------+
|                       Root  Dispersion                         | 2
+----------------------------------------------------------------+
|                     Reference Identifier                       | 3
+----------------------------------------------------------------+
|                    Reference Timestamp (64)                    | 4
+----------------------------------------------------------------+
|                    Originate Timestamp (64)                    | 6
+----------------------------------------------------------------+
|                     Receive Timestamp (64)                     | 8
+----------------------------------------------------------------+
|                     Transmit Timestamp (64)                    | 10
+----------------------------------------------------------------+
|                 Key Identifier (optional) (32)                 | 12
+----------------------------------------------------------------+
|                 Message Digest (optional) (128)                | 13+
+----------------------------------------------------------------+

I used the reference in the first link, but the same is available in the NTP RFC document (latest version - v4). What you can see is a 48 byte packet, with a number of fields.

Turns out, we only need the first few for making a request:

  1. The first 2 bits are used for some sort of Leap Indicator. That sounds cool - but we don't need it now, so we can set it to 0.
  2. The next 3 bits [2-5), are a version number indicator. We can set it to version 3 (011) or 4 (100). We went with v3 just because that is what the python implementation had.
  3. The next 3 bits [5-8) are used as Mode. Not sure what all values hold for this, but from both the examples it is clear that it has to be 3 (011) for a client request.

That gives us 00011011 as the first byte of the packet. For a real NTP client, we would be interested in setting other fields with the current timestamp since that is used in the delay calculations to get super accurate timestamps. I only care in getting second level values here, so, we can leave all other fields empty.

So, a very simple Python snippet is as follows: 

import socket

# create a UDP socket
client = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

# create the packet data
# >>> hex(0b00011011)
# '0x1b'
# 47 null bytes
data = '\x1b' + 47*'\0'

client.sendto(data, ("time.nplindia.org", 123))

Now, we look at the response on Wireshark - a 90 byte packet of which 48 bytes are the actual NTP payload. It looks like this: 

Network Time Protocol (NTP Version 3, server)
    Flags: 0x1c, Leap Indicator: no warning, Version number: NTP Version 3, Mode: server
        00.. .... = Leap Indicator: no warning (0)
        ..01 1... = Version number: NTP Version 3 (3)
        .... .100 = Mode: server (4)
    Peer Clock Stratum: primary reference (1)
    Peer Polling Interval: invalid (3)
    Peer Clock Precision: 0.000004 sec
    Root Delay: 0 seconds
    Root Dispersion: 0.001068115234375 seconds
    Reference ID: Generic pulse-per-second
    Reference Timestamp: May  5, 2023 07:01:15.357278010 UTC
    Origin Timestamp: Jan  1, 1970 00:00:00.000000000 UTC
    Receive Timestamp: May  5, 2023 07:01:20.628566043 UTC
    Transmit Timestamp: May  5, 2023 07:01:20.628696752 UTC

Note how the Mode is set to 4, Server.

That is nice. For our purposes, we can use one of the last 2 values - Receive or Transmit as is. In reality, clients are supposed to do some calculations as shown in the Wikipedia page.

What about these timestamps - Wireshark decodes them nicely for us, but how do we do it?

Turns out there are few complexities with the timestamps. Let us say, we pick the Receive Timestamp which is the 32-40 bytes, ie 8 bytes. But how is this decoded into the timestamp? Turns out:

  1. It is a binary fixed-point representation with the first 32 bits representing seconds and the other 32 bits representing fractional seconds.
  2. It counts time since January 1, 1990 (and not 1970 as one would have imagined) which is a bit of unexpected surprise!

So, all we need to do to get the timestamp out is: 

resp, _ = client.recvfrom(48)

ans = struct.unpack("!I", resp[32:36])[0] - 2208988800

That last line needs a bit of explanation.

  1. We focus on the 4 bytes of interest from 32 to 36.
  2. We unpack it using the format "!I". The ! means to use net-order and the I stands for unsigned integer.
  3. We now have seconds since 1900. By subtracting out this magic number, which is the number of seconds between 1900 and 1970, we have the seconds since epoch.

A bit of a demo below:

I just realized that I didn't update my local time yet, in all this. Let me go do that now!

Some useful links: