The other side of the moon

I spent quite some time a couple of months ago working on timezone support for mPulse and thought I should document the insanity, but never quite got around to it. Then there was this post on hacker news about reading a user's timezone in JavaScript and using that to display the right time. That post brought back a flood of horrific memories, prompting me to put my thoughts down.

First, while Trevor's post has a good hack to display a time in the user's current timezone, that hack works in only one case -- displaying the current time to the user in their device's timezone.

If you've worked with timezones and front end development for a while, this is probably the first hack you'll think up.  It turns out that in most cases, this is insufficient.

We'll first look at the problems with this approach, and then look at the requirements for proper timezone support.

Problems

• The user's device timezone is not always correct. Some users fix their device timezone to their home time even when they're travelling, however the information you need to display may be pertinent for the location where they are right now.
• On the other hand, the user may have their device set to automatically update timezone, but they actually want to see times in their home time (because, for example, that's when they call home, or have their calendar app configured).
• The timezone offset (which is what you actually get from JavaScript), only tells you the offset from UTC for "right now". This information is irrelevant if you need to display a time that is not now, because daylight saving rules may come into effect.
• You cannot use a lookup table for offset to timezone, because there isn't a one-to-one mapping between offset and timezone. It's a many-to-many mapping, and it changes.

Second attempt

A second attempt might be to figure out the timezone name by parsing the JavaScript Date.toString() output. This was my second attempt when writing the strftime function for the YUI Library.

I did this study in 2008, and it turns out that browsers are pretty inconsistent wrt Date.toString() output.

Requirements

Ok, before going into this, read this post on stackoverflow about daylight saving time and timezones.

So, what we need is the ability to do the following:

1. store any date or range of dates.
2. display a date in any timezone that makes sense for the user, and/or the event(s) being displayed, and/or the environment.
3. display date ranges that may cross a timezone boundary.
4. display a historic date in a historic timezone that may have changed due to political decisions.

The first requirement should be pretty straightforward. We'd like to store dates, and the best way is really a unix timestamp or an ISO8601 date. I prefer the latter because it takes into account leap seconds as well (unix timestamps are leap second agnostic [1],[2]). I also always use Zulu time for an ISO8601 date.

This is not sufficient, however. We also need to store the timezone name of the event. This is so that we can display historic events in the timezone they originally occurred in, even if the definition of that timezone changes. This comes from the Olson Database.

With these two pieces of information (event date/time & event timezone name), we can render the date in several ways... the original event date/time, the event date/time relative to the user's current timezone, etc.

We also need to handle date ranges. This could be something like your spring vacation, that just happened to cross several timezones because you left San Francisco on March 8th, flew to the UK, stayed there until April 7th, and then flew back. Your flight departure from SFO is in Pacific Standard Time and your arrival at LHR is in British Standard Time. Your departure from LHR is in British Daylight Time, and your arrival at SFO is in Pacific Daylight Time.

What's most important is that you display these dates in their specific timezones regardless of where the user actually is.

So should we guess or ask the user what they want?

By all means guess at what the user's timezone might be. Use a combination of GeoIP + JavaScript timezone offset to figure out where they might be (note that both of these could be wrong), but give them the option to specify the timezone that they care about.

Also, when displaying event dates, use a date local to the event, but use JavaScript to allow the user an easy way to flip it to their local timezone if they like. That's progressive enhancement.

What else shouldn't we do?

Don't try and guess the user's language or preferred currency from their current location. Always ask and store their response.

Click to help me test boomerang (opens google.com in a new window).

Wrote this as an answer to a question on StackOverflow, but I think it was one of my better ones, so am reproducing it here.

There are four different times you should consider:

1. Event time: eg, the time when an international sporting event happens, or a coronation/death/etc. This is dependent on the timezone of the event and not of the viewer.
2. Television time: eg, a particular TV show is broadcast at 9pm local time all around the world. Important when thinking about publishing the results (of say American Idol) on your website
3. Relative time: eg: This question has an open bounty closing in 21 hours. This is easy to display
4. Recurring time: eg: A TV show is on every Monday at 9pm, even when DST changes.

There is also Historic/alternate time. These are annoying because they may not map back to standard time. Eg: Julian dates, dates according to a Lunar calendar on Saturn, The Klingon calendar.

Storing start/end timestamps in UTC works well. For 1, you need an event timezone name + offset stored along with the event. For 2, you need a local time identifier stored with each region and a local timezone name + offset stored for every viewer (it's possible to derive this from the IP if you're in a crunch). For 3, store in UTC seconds and no need for timezones. 4 is a special case of 1 or 2 depending on whether it's a global or a local event, but you also need to store a created at timestamp so you can tell if a timezone definition changed before or after this event was created. This is necessary if you need to show historic data.

Storing times

• Always store time in UTC
• Convert to local time on display (local being defined by the user looking at the data)
• When storing a timezone, you need the name, timestamp and the offset. This is required because governments sometimes change the meanings of their timezones (eg: the US govt changed DST dates), and your application needs to handle things gracefully... eg: The exact timestamp when episodes of LOST showed both before and after DST rules changed.

Offsets and names

An example of the above would be:

The soccer world cup finals game happened in South Africa (UTC+2--SAST) on July 11, 2010 at 19:00 UTC.

With this information, we can historically determine the exact time when the 2010 WCS finals took place even if the South African timezone definition changes, and be able to display that to viewers in their local timezone at the time when they query the database.
System Time

You also need to keep your OS, database and application tzdata files in sync, both with each other, and with the rest of the world, and test extensively when you upgrade. It's not unheard of that a third party app that you depend on did not handle a TZ change correctly.

Make sure hardware clocks are set to UTC, and if you're running servers around the world, make sure they're OSes are configured to use UTC as well. This becomes apparent when you need to copy hourly rotated apache log files from servers in multiple timezones. Sorting them by filename only works if all files are named with the same timezone. It also means that you don't have to do date math in your head when you ssh from one box to another and need to compare timestamps.

Also, run ntpd on all boxes.

Clients

Never trust the timestamp you get from a client machine as valid. For example, the Date: HTTP headers, or a javascript Date.getTime() call. These are fine when used as opaque identifiers, or when doing date math during a single session on the same client, but don't try to cross-reference these values with something you have on the server. Your clients don't run NTP, and may not necessarily have a working battery for their BIOS clock.

Trivia

Finally, governments will sometimes do very weird things:

Standard time in the Netherlands was exactly 19 minutes and 32.13 seconds ahead of UTC by law from 1909-05-01 through 1937-06-30. This time zone cannot be represented exactly using the HH:MM format.

Since my last post on javascript dates, I've been doing a lot more reading and testing. This post is a summary of what I've found.

I read up a few RFCs and ISO specifications. In particular, RFC2822: Internet Message Format, and RFC3339: Date and Time on the Internet: Timestamps, which summarises ISO8601: An internet standard for date and time (Wikipedia link). Below is a summary of what these specifications state.

RFC2822 lists two date formats, in sections 3.3 and 4.3. 3.3 is current while 4.3 is obsolete. RFC3339 lists a third date format based on ISO8601. All these specifications are for date-time representations for use on the Internet.

RFC3339:

2008-09-23T17:39:44-07:00

RFC2822:

Tue, 23 Sep 2008 17:39:44 -0700

(obsolete) Tue, 23 Sep 2008 17:39:44 PDT (might also include 2 digit year)

Both specifications state that the time zone code SHOULD NOT be used, and only the offset should be used. The colon used in the time zone offset is optional in ISO8601, but not in RFC3339. The time zone may be replaced with Z, indicating Zulu time (UTC+0000).

Additionally, RFC2822 lists 10 codes in the obsolete: UT, GMT, EST, EDT, CST, CDT, MST, MDT, PST, PDT. Military time zones (A-Z) are also included in the obsolete list. While other time zone codes have been used by various implementations, they are not defined by a common specification, and SHOULD NOT be used.

My tests with the Date.toString() method have shown the following:

Internet Explorer returns the time zone code for the eight US time zones listed in section 4.3. of RFC2822, and the offset for all other time zones using the UTC+/-HHMM format. The offset is NOT included for the eight US time zones specified above. This is in line with RFC822, but not 2822, which obsoletes 822.

Examples:

PST:

Thu Sep 18 16:47:36 PDT 2008

IST:

Fri Sep 19 10:11:38 UTC+0530 2008

Firefox returns the offset for all time zones, and a time zone string in parentheses for some time zones. The string is different on different platforms:

Windows:

full timezone name, ie, Pacific Standard Time

Linux & Mac OS X:

timezone code, ie, PST

examples:

Windows/Taiwan

Tue Sep 23 2008 10:14:10 GMT+0800

Windows/India

Tue Sep 23 2008 09:42:49 GMT+0530 (India Standard Time)

Mac OS X/US West Coast

Tue Sep 23 2008 17:57:04 GMT-0700 (PDT)

Linux/India

Tue Sep 23 2008 09:42:49 GMT+0530 (IST)

Safari matches the results for Firefox on Mac OS X.

Both Safari and Firefox match the specification by including the offset. The time zone code that they return is considered auxiliary information for display purpose only.

Opera consistently uses the UTC+/-HHMM format on all platforms, and never includes a time zone code. This is probably the strictest adherence to the specification.

So, what does all this mean? Well, to me, it means that I shouldn't spend any more time searching for a browser independent way to pull out the time zone code. It also means that strftime does not provide an easy way to print out an RFC3339 date with the colon in the time zone offset, but we'll figure something out for that :)

Finally, a bit of trivia...

Standard time in the Netherlands was exactly 19 minutes and 32.13 seconds ahead of UTC by law from 1909-05-01 through 1937-06-30. This time zone cannot be represented exactly using the HH:MM format.

Short URL: http://tr.im/dateformats

Ever since I built my javascript strftime implementation, I've been playing around with dates and date formatting in javascript. One of the things that users of the library have been happy about is the ease with which dates can be localised using the library, and it's this particular feature that's been frustrating me a lot when dealing with time zones. Let me get this out of my head.

The javascript engine of various browsers stringify dates in subtly different ways, but different enough to mess up time zone parsing.

Here's what I'm talking about. Tell me what your browser says when you click the following two links:

• Date.toString()
• timezone

Try this snippet of code in your favourite browsers:

var d = new Date();
alert(d.toString());
alert(d.toLocaleString());

I've seen different results on Firefox, IE, Opera and Safari. Firefox is the worst offender with different results across platforms. The following are my results:

Firefox 2 & 3 on Linux and Mac OS X, and Safari on Mac OS X

Thu Sep 18 2008 19:20:23 GMT-0700 (PDT)

Firefox 2 & 3 on Windows

Thu Sep 18 2008 19:20:23 GMT-0700 (Pacific Daylight Time)

Google Chrome

Thu Sep 18 2008 19:20:23 GMT-0700 (Pacific Daylight Time) However it displays GST instead of GMT if your timezone is set to GMT

Opera 9.52 on Linux and Mac OS X

Thu Sep 18 2008 19:20:23 GMT-0700

IE 6, 7, 8 on Windows

Thu Sep 18 19:20:23 PDT 2008

I've only tested this in the PDT time zone, but the results are diverse enough to make parsing out the time zone annoying (except for Opera, where it's impossible). This is what I came up with:

d.toString().replace(/^.*:\d\d( GMT[+-]\d+)? \(?([A-Za-z ]+)\)?\d*$/, '$2').replace(/[a-z ]/g, '');

I derived this regex over a couple of iterations. It started out taking care of the Firefox on Mac & Linux and Safari case:

d.toString().replace(/^.* \(([^)]+)\)$/, '$1');

Next, I added support for Firefox on Windows by getting rid of all lowercase letters and spaces:

d.toString().replace(/^.* \(([A-Za-z ]+)\)$/, '$1').replace(/[a-z ]/g, '');

Finally, I added in support for IE, by checking for a string of characters that fell between seconds and year:

d.toString().replace(/^.*:\d\d( GMT[+-]\d+)? \(?([A-Za-z ]+)\)?\d*$/, '$2').replace(/[a-z ]/g, '');

This is by no means fool proof. It could break with time zones and browsers that I haven't encountered yet, so I'm asking everyone for help in figuring this out.

Short of asking all browser authors to be consistent in their Date.toString() implementations, how do I figure out the timezone string for a date? I've even thought about building a table of offsets to timezone strings, but there isn't a one to one mapping for that.

I've also noticed that the timezone is always displayed in English, even when I change my system locale. Perhaps this is different for you. Let me know.

And yo, Google, what's up with Greenwich Standard Time?

Short URL: http://tr.im/jsinconsistentdate