Back to Basics: Daylight Savings Time bugs strike again with SetLastModified
_3.jpg "CC BY-NC 2.0 Creative Commons Clock Photo via Flickr ©Thomas Hawk")
No matter how well you know a topic, or a codebase, it's never to late (or early) to get nailed by a latest bug a over a half-decade old.
DasBlog, the ASP.NET 2 blog engine that powers this blog, is done. It's not dead, but it's done. It's very stable. We had some commits last year, and I committed a bug fix in February, but it's really well understood and very baked. My blog hasn't been down for traffic spike reasons in literally years as DasBlog scales nicely on a single machine.
It was 10:51pm PDT (that's Pacific Daylight Time) and I was writing a blog post about the clocks in my house, given that PST (that's Pacific Standard Time) was switching over soon. I wrote it up in Windows Live Writer, posted it to my blog, then hit Hanselman.com to check it out.
Bam. 404.
What? 404? Nonsense. Refresh.
404.
*heart in chest* Have I been hacked? What's going on? OK, to the logs!
l2 time 2011-11-06T05:36:31 code 1 message Error:System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
Parameter name: utcDate
at System.Web.HttpCacnhePolicy.UtcSetLastModified(DateTime utcDate)
at System.Web.HttpCachePolicy.SetLastModified(DateTime date)
at newtelligence.DasBlog.Web.Core.SiteUtilities.GetStatusNotModified(DateTime latest) in C:\dev\DasBlog\source\newtelligence.DasBlog.Web.Core\SiteUtilities.cs:line 1253
at newtelligence.DasBlog.Web.Core.SharedBasePage.NotModified(EntryCollection entryCollection) in C:\dev\DasBlog\source\newtelligence.DasBlog.Web.Core\SharedBasePage.cs:line 1182
at newtelligence.DasBlog.Web.Core.SharedBasePage.Page_Load(Object sender, EventArgs e) in C:\dev\DasBlog\source\newtelligence.DasBlog.Web.Core\SharedBasePage.cs:line 1213
at System.EventHandler.Invoke(Object sender, EventArgs e)
at System.Web.UI.Control.OnLoad(EventArgs e)
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) while processing http://www.hanselman.com/blog/default.aspx.
What's going on? Out of range? What's out of range. Ok, my site is down for the first time in years. I must have messed it up with my clock post. I'll delete that. OK, delete. Whew.
Refresh.
404.
WHAT?!?
Logs, same error, now the file is a meg and growing, as this messages is happening of hundreds of times a minute. OK, to the code!
UtcSetLastModified is used for setting cache-specific HTTP headers and for controlling the ASP.NET page output cache. It lets me tell HTTP that something hasn't been modified since a certain time. I've got a utility that figures out which post was the last modified or most recently had comments modified, then I tell the home page, then the browser, so everyone can decide if there is fresh content or not.
public DateTime GetLatestModifedEntryDateTime(IBlogDataService dataService, EntryCollection entries)
{
//figure out if send a 304 Not Modified or not...
return latest //the lastTime anything interesting happened.
}
In the BasePage we ask ourselves, can we avoid work and give a 304?
//Can we get away with an "if-not-modified" header?
if (SiteUtilities.GetStatusNotModified(SiteUtilities.GetLatestModifedEntryDateTime(dataService, entryCollection)))
{
//snip
}
However, note that I'm have to call SetLastModified though. Seems that UtcSetLastModified is private. (Why?) When I call SetLastModified it does this:
public void SetLastModified(DateTime date)
{
DateTime utcDate = DateTimeUtil.ConvertToUniversalTime(date);
this.UtcSetLastModified(utcDate);
}
Um, OK. Lame. So that means I have to work in local time. I retrieve dates and convert them ToLocalTime().
At this point, you might say, Oh, I get it, he's called ToLocalTime() too many times and double converted his times. That's what I thought. However, after .NET 2 that is possible.
The value returned by the conversion is a DateTime whose Kind property always returns Local. Consequently, a valid result is returned even if ToLocalTime is applied repeatedly to the same DateTime.
But. We originally wrote DasBlog in .NET 1.1 first and MOVED it to .NET 2 some years later. I suspect that I'm actually counting on some incorrect behavior deep in own our (Clemens Vasters and mine) TimeZone and Data Access code that worked with that latent incorrect behavior (overconverting DateTimes to local time) and now that's not happening. And hasn't been happening for four years.
Hopefully you can see where this is going.
It seems a comment came in around 5:36am GMT or 10:36pm PDT which is 1:36am EST. That become the new Last Modified Date. At some point we an hour was added in conversion as PDT wasn't PST yet but EDT was EST.
Your brain exploded yet? Hate Daylight Saving Time? Ya, me too.
Anyway, that DateTime became 2:36am EST rather than 1:36am. Problem is, 2:36am EST is/was the future as 6:46 GMT hadn't happened yet.
A sloppy 5 year old bug that has been happening for an hour each year that was likely always there but counted on 10 year old framework code that was fixed 7 years ago. Got Unit Tests for DST? I don't.
My server is in the future, but actually not as far in the future as it usually is. My server in on the East Coast and it was 1:51am. However, the reasons my posts sometimes look like they are from the future, is I store everything in the neutral UTC/GMT zone, so it was 5:51am the next day on my file system.
Moral of the story?
I need to confirm that my server is on GMT time and that none of my storage code is affected my Daylight Saving Time.
Phrased differently, don't use DateTime.Now for ANY date calculations or to store anything. Use DateTime.UTCNow and be aware that some methods will freak out if you send them future dates, as they should. Avoid doing ANYTHING in local time until that last second when you show the DateTime to the user.
In my case, in the nine minutes it took to debug this, it resolved itself. The future became the present and the future last modified DateTime became valid. Is there a bug? There sure it, at least, there is for an hour, once a year. Now the real question is, do I fix it and possibly break something that works the other 8759 hours in a year. Hm, that IS still four 9's of uptime. (Ya, I know I need to fix it.)
"My code has no bugs, it runs exactly as it was written." - Some famous programmer
Until next year. ;)
About Scott
Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.
About Newsletter
Simon
That's why, in my current VPS server, I have followed Tatham Oddie's advice, and set the server local time to UTC, as in:
http://blog.tatham.oddie.com.au/2007/07/30/which-time-zone-should-i-set-for-my-server/
Where I work, we have a mainframe that is set to local time. Because of that, techs have to shut it down to manually switch it to DST and back. They also have to keep it offline for an hour in order to avoid timestamp issues when turning back the clock.
One hour of the year this happens: The local times and daylight saving time changing can affect the programming that lies behind the website pages. If you experience this problem, come back on the next hour. XX:00 and all will be well.
Thus proving that some programmers can be practical and not anal-retentive.
Having a type that actually keeps the offset around on its own instead of doing it yourself (typically by keeping everything in UTC, so you're effectively keeping around an offset of 0) makes things so much simpler and prevents entire classes of bugs, especially around DST.
BCL type: http://msdn.microsoft.com/en-us/library/system.datetimeoffset.aspx
SQL type: http://msdn.microsoft.com/en-us/library/bb630289.aspx
The BCL team's blog post discussing the new type:
http://blogs.msdn.com/b/bclteam/archive/2007/06/14/datetimeoffset-a-new-datetime-structure-in-net-3-5-justin-van-patten.aspx
The one caveat is that for others using WCF RIA Services, support for DateTimeOffset is coming in 1.0 SP2.
http://channel9.msdn.com/Shows/SilverlightTV/Silverlight-TV-74-Whats-New-in-RIA-Services
GMT never changes and is, to within fractions of a second I believe, the same as UTC. In the UK our 'daylight' time is British Summer Time (BST) which is GMT/UTC+1.
DST <==> not DST bites in many ways. Example: Neudesic has not reset one of more servers and appears to have the auto reset turned off so replies are getting messed up ... worst case setting is replies during the 01:00 to 02:00 time frame where DST overlaps with not DST giving the impression that members have psychic ability and can answer questions before they have been asked.
Use your extra hour wisely!! B-)
regards ~~ gerry (lowry)
If you find out about downtime by manually checking the web -means you had more downtime than should have.
2) We keep all calculations in UTC too. That helps a lot.
3) Switching to Daylight Savings time every year is plain wrong.
4) My personal preference would be not only stop switch to/from Daylight Savings Time, but use UTC everywhere.
So instead of thinking about timezones, we would think that West Coast start their work day at 17:00 as opposing to East Coast starting work day at 14:00.
I understand that society is not ready for that yet. But may be in ~50 years that would happen.
Answer: no, you "fix it without breaking something that works the other [8783] 8759 hours in a [leap] year."
You've an extra day because 2012 is a leap year ... so now you have something to do for next February 29th!
The devil is in the details; the solution is simplistic: you try/catch System.ArgumentOutOfRangeException and in your catch exception check for the demon DST overlap and then decide how you can gracefully ignore it.
or, an even better and simpler solution that requires zero code changes ... during the TWO hour time frame that occurs during than annual DST ==> not DST switch, shut your site down for TWO hours of maintenance.
TIMTOWTDI =. there is more than one way to do it
regards, gerry (lowry)
So I went to the home page (dasblog.info). Thought I would have a look at the forums via the link at the top of the page =>404.
Don't worry too much about an error for one hour of the year, there are more prosaic problems.
I couldn't see a link to report site problems, hence the post in this forum.
http://www.kyivpost.com/news/nation/detail/113166/
My block post on retrieving DateTime as UTC by default from Entity Framework: http://www.aaroncoleman.net/post/2011/06/16/Forcing-Entity-Framework-to-mark-DateTime-fields-at-UTC.aspx
Would love to see a global config setting in ASP.NET that defaults EF to DateTime.Kind = UTC
Oh, also what you mentioned in this article, is my very reason to almost build everything from scratch.
Thanks.
Anyone up for writing a short tutorial?
I'm interested in the architecture people would suggest, and why.
Lets say I have a asp.net mvc app hosted on a server in the East Coast and used all over the world.
But, the application never takes the local time from any of the client machines and never uses the local time from the client to store, update, cache, log ,etc.
The application always uses the server's local time, why would I have a problem ?
Comments are closed.