Scott Hanselman

browse by category or date

Video: Windows Browser Speed Shootout - IE7, Firefox2, Opera9, Safari for Windows Beta 3

June 12, '07 Comments [27] Posted in Musings | Reviews | Tools
Sponsored By
There is a lot of good discussion about the Safari for Windows release and my, as well as others (including Apple's), assertion that it's fast. Or at least it feels fast. A few folks said that it might feel fast because Steve Jobs said it's fast. 

So, I decided over lunch to do a non-scientific (but using common sense) test. I started by loading up each browser and resizing them to exactly 800x600. I used Camtasia to record them, independently, loading hanselman.com, apple.com, cnn.com, msdn.com, sun.com and craigslist.org.

I then downloaded a free version of Sony Vegas and made a 1600x1200 (four quadrants of 800x600) and put each site's loading in their appropriate quadrant. I started the timer (and start frame) when I press Enter in the Address Bar and stopped it when I saw a "done" indication from the browser. For some that was the word "done" in the Status Bar, for others it was a progress bar finishing and disappearing.

Browser Speed

I did the whole thing, capture and edit in Raw 30fps and didn't make it smaller until the final Render. I deleted browser caches and pre-cached DNS for each. Each browser's file was about 1.2 gigs each.

Here's a table of the results:

  Desc FF2 IE7 Opera9 Safari3
hanselman.com heavy, css, javascript 3.604 5.939 4.872 4.872
apple.com light, lots of images 3.670 4.337 4.27 1.134
cnn.com heavy, javascript, many ads 5.339 2.936 3.337 2.002
msdn.com very heavy 6.74 11.946 12.88 2.67
sun.com light, big image 4.938 1.868 3.336 1.801
craigslist.org very light, nearly all text 1.602 0.868 0.668 0.668

You can check the video for yourself, but the nutshell conclusion for the non-Safari browsers is "it depends on the site," but on my computer, Safari on Windows is not slow. The question is, are you willing to put up with a Mac-like interface, wonky text anti-aliasing and some weird rendering, without your FireFox plugins? I'm going to give it a try and see how it goes. It's a lot to ask.

Also, Craigslist just generally rocks. Enjoy.

Technorati Tags: , , , ,

« Microsoft Silverlight in Safari for Wind... | Blog Home | Six Essential Language Agnostic Programm... »

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Comments [27]
Tuesday, June 12, 2007 10:58:44 PM UTC
Wow. 12 seconds for IE to load MSDN, and Safari does it in about 3. Crazy.
John Batdorf
Tuesday, June 12, 2007 11:15:38 PM UTC
Are we looking at the same video? Safari is the fastest or tied for faster on all but your site. Saying it "depends on the site" doesn't seem far to how fast Safari is right now. Also, a table with all the results would be nice.
Dave Murdock
Tuesday, June 12, 2007 11:23:30 PM UTC
it would be cool to get the veritest ibench suite to do our own tests on our own sites and networks -- i couldn't find it...anyone know where it is?
timheuer
Tuesday, June 12, 2007 11:34:22 PM UTC
In IE7, did you have the phishing filter on? Any other add-ons turned on in any of the browsers?

One point that occurs to me though, unfortunately this is hard to do a truly empirical test with everything equal... you can't even be sure the same code paths are being executed (on server or client side). In other words, I could write some server side code that generated my HTML and then called Thread.Sleep() for a few seconds depending on the browser. I'm not saying this is what's happening, I'm saying we don't know. We can't tell if everything we are observing is happening within the browser process itself, and even if it is (i.e., JavaScript) the same functionality may not necessarily be being used across all browsers.
Tim K
Tuesday, June 12, 2007 11:36:46 PM UTC
Tim K - The IE7 Phishing filter was OFF in this test.

I hear what you're saying about the server-side path, but I just loaded the home pages, and they all looked the same, but yes, different ads might have been served.
Scott Hanselman
Tuesday, June 12, 2007 11:43:00 PM UTC
More specifically, I am wondering what kind of "up-level" HTML is rendered on MSDN. Safari on Windows should be a down-level browser by default (I don't know that--I'm guessing). My basic point is that in general, MSDN should be the slowest in IE because Microsoft is going to use the browser to the fullest extent possible.
Tim K
Tuesday, June 12, 2007 11:44:50 PM UTC
Excellent point...of these sites MSDN would be the most likely to do just that. Possibly not the best site to choose, but there you go.
Scott Hanselman
Wednesday, June 13, 2007 12:12:59 AM UTC
I believe you should also include the time spent to fire up the browser in the first place. Got some data on the load times for each?
anon
Wednesday, June 13, 2007 12:18:25 AM UTC
I don't agree with the "wonky text anti-aliasing", implying bad. Jeff Atwood was on about that yesterday. I think it is personal preference. Back when I started using OS X in 2003, that was one of the first things I noticed, fonts were better on OS X than XP to my eyes/taste, no matter how much ClearType tweaking I did. I haven't spent a tremendous amount of time in Vista to see if my opinion changes, maybe that's what you guys are talking about, but the time I have spent in Vista hasn't changed my opinion.
Dave Murdock
Wednesday, June 13, 2007 12:21:35 AM UTC
What you could do for a bit more thorough test would be to run it again with a script (Selenium, NUnitASP, Watir) and have it browse a few pages on each site. It could run each of the websites in sequence, perhaps multiple times to even out the variations due to ads, etc. But I would be interested in recording the timings and page sizes with the test script today, in 3 months, 6 months, etc. If I were to do it I would use Selenium with the SeleniumIDE extension for FireFox but the Watir framework may have a few additional features that would help gather more detail.

I would also like to know how you used Sony Vegas to piece this together. I have it downloaded and I will see if I can put something together.
Brennan Stehling
Wednesday, June 13, 2007 1:43:02 AM UTC
Based on your informal test, you've done a good job proving Safari isn't slow, but I'd be more convinced if the tests were executed multiple times and averages for each site were used, particularly for MSDN. In my experience, MSDN's performance fluctuates wildly throughout the day. Safari may very well be faster, but a wider sample is needed to make up for possible statistical anomalies.

Once we've answered the question about the relative speed of Safari, the more important question is whether or not that's a compelling enough reason to use it. Mac fanboys don't even use it (at least, none of the Mac fanboys I know...).
Cam Soper
Wednesday, June 13, 2007 11:45:49 AM UTC
Flawed test, sorry.
I just tested the loading of msdn.com with FF2 and Opera9 (latest versions). The results: both load the page in 9 seconds.
I really don't know where did you get the 6 seconds difference. I tested several times and cleared the cache and all.
toby
Wednesday, June 13, 2007 4:39:57 PM UTC
Toby - I just report the news. It's true, though, that MANY folks are having huge problems with Safari not working well, or not at all, on their machines. I just filmed my experience. Not sure why you're seeing what you're seeing.
Scott Hanselman
Wednesday, June 13, 2007 4:40:38 PM UTC
Ars Technica posted a good comparison of Safari vs. IE vs. Firefox, but they didn't do any in-depth speed tests beyond saying that Safari "offers slightly faster page loading."
Cam Soper
Wednesday, June 13, 2007 6:18:45 PM UTC
I'm not sure on what you're saying here... i mean, even if it's faster, it's clear that it can't render many pages correctly. For instance, it looks like it has lots of problems with anchors. what's the purpose of being quick it it doesn't render the pages correctly?
Luis Abreu
Wednesday, June 13, 2007 6:21:30 PM UTC
Luis - I'm just saying it's fast. If the they get the rendering right, then it raises all of our expectations of quickness. I have a Core2Duo that takes 5 seconds to "draw" a page. How much of that is network time? DNS? Rendering? Let's get rendering to ~0 seconds. If they do that, it'll make other browsers faster.
Scott Hanselman
Wednesday, June 13, 2007 6:33:39 PM UTC
Scott,

I am curious how Safari is setup on your machine. For some reason and it might just be me, but in the sites that I visit frequently are usually always really slow. A good example of this is slashdot, at least on my computer.
Brandon K.
Wednesday, June 13, 2007 6:47:02 PM UTC
Something has to be wrong here, Scott. I just duplicated your msdn.com and apple.com test with IE7 and Safari beta here at home on a pristine, clean install of Windows Vista with caches and history cleared. There is nowhere close to a delta of 11.496 vs. 2.67 and 4.337 vs. 1.134 respectively. They are equivalent as far as my naked eye is concerned.

You might also want to take a look at http://msdn2.microsoft.com/en-us/library/default.aspx in both browsers as well. Safari exhibits the same result on my Mac, so can't be because it's a beta.
Charlie Owen
Wednesday, June 13, 2007 8:17:45 PM UTC
@Charlie - yes, it is weird. Of course, it's beta, and it's pretty clear that everyone is seeing VERY different behaviors one their machines. Could have been a (consistent) DNS glitch on my side.

@Brandon - I didn't change any of the Safari defaults.
Scott Hanselman
Wednesday, June 13, 2007 9:08:21 PM UTC
Scott,

Thanks, there must be something really screwy then on my machine. So far I haven't been able to see anything that is really impressive with Safari. The one site that I think is really odd is how www.codezone.com looks on Safari when compared to FF2 and IE7. Also how difficult was it for you to use Sony Vegas?
Brandon K
Thursday, June 14, 2007 12:01:18 AM UTC
Did you tweak any of the FF2 about:config settings, or was this a vanilla install?
Jeremy Saldate
Thursday, June 14, 2007 12:03:29 AM UTC
Vanilla. No faster fox.
Scott Hanselman
Thursday, June 14, 2007 12:58:52 PM UTC
Thanks for putting this together. Tim's comment about ads is perhaps the most relevant. Downlevel html probably had a lot to do with the major lag you're seeing on IE for any news site.

I work on the software team at Newsweek.com, and there are some major difference between the JavaScript and ad packages that we serve for Opera and Safari when compared to IE or FF. I think the the major winner of this study is really FF. All of those sites play much more fairly with IE and FF then they do with Safari to the extent of not serving Flash for ad packages or not using Ajax for interface manipulation. Handing down those libraries and instantiating DOM objects when the page loads is going to hold open the loading state much longer.

I wonder what the benchmarks would look like if you downloaded the site html locally and ran all these test locally. That would ensure that your content was exactly the same. However, it would require a much more accurate, automatic way of stopping and starting the timer because they would the content would load much faster. That would also get rid of any unfair abnomolies with DNS or network time.

------------------
Jeff Devinian
My web site
My non profit software
Jeff Devinian
Friday, June 15, 2007 2:08:04 AM UTC
I use IE and FF. Even if Safari is faster, I am not planning to use a third browser. Two is enough. As a web developer, developing sites that work well with IE and FF is already a pain in itself. I really hope I don't have to deal with Safari in the future
Abdu
Friday, June 15, 2007 11:21:38 PM UTC
I agree MSDN is a bad choice as it will most probably serve different (usable) content to different browsers (additionally Opera uses BrowserJS to make msdn to work properly). Even worse is the Wired test using just Gmail and Google Calender. I don't know about the rest of the pages you use in your benchmark but your benchmark is the best benchmark with Safari for Windows I know until now.
For the flaws of iBench used by Apple I recommend "Safari and page load timing" by Mark "Tarquin" Wilton-Jones (in short: iBench and other Tools use JS onload as sign that the page is completly rendered while in Safari "onload fires before the page has been displayed, before layout has been calculated, before any costly reflows have taken place"). You can find some hints about browser benchmarking there as well as in Tarquin's Browser speed comparisons (although the tests themselves are said to be retired).
ResearchWizard
Saturday, June 16, 2007 5:50:12 AM UTC
ie7 is suck
ie7
Tuesday, July 31, 2007 2:17:08 AM UTC
good
ajothirathna
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.