Somewhere in late 1997, early 1998 (as I recall) I was working at a place called 800.com. Me and three guys (Joe Tillotson, Javan Smith and Patrick Cauldwell) from our consulting firm were helping create the storefront, shopping cart and business systems. We did it all in COM and Classic ASP. I did most of the front-end ASP, HTML, cut graphics, did scale, browser compat, the whole thing. Javan ended up working there full time for some years as I recall.

It was deep in the beginning of the first boom. We were sleeping with our servers. The place had no offices yet, and we were sitting on the stairs of the Tyee Group in Portland. March 1, 2002 800.com sold to Circuit City and disappeared. I still say they could have been like Amazon, but it just didn't work out.

Anyway, the CEO Greg Drew and his folks had this crazy idea for a promotion. It was simple, 3 DVDs for $1. That's it. Free shipping and everything. It was insane.

In the same vein of crazy promotions, SparkFun Electronics had "Free Day" today. It was also a brilliant promotional idea. They offered $100 per person, until they hit $100,000. Of course, that's only 1000 people but you don't say that part. ;) Every geek on the planet had their finger on the "Buy" button and word spread like wildfire on Twitter. The whole thing was over in 1h 44m 50s, according to their website. I spent an hour trying to check out my cart, and only succeeded in loading a page twice. Bummer.

With 800.com, our 3 DVDs for $1 promotion lasted longer, a few days as I recall, and word travelled much slower, on USENet and Email. Barbarians. ;)

Here's some of the things we did to optimize the 800.com site for this massive traffic surge in that pre-cloud era. It worked for us. Perhaps SparkFun did, or could have, done similar things.

Think about the Ratio of Reads to Writes

When people visit your site, are they mostly reading data, like browsing a product catalog? Or are they mostly writing data, like putting something in a shopping cart and trying to check out?

There's basically these four types of data on a web site: resource or catalog data, infrequently-used reference data, user session data, and clickstream or activity oriented data. Each is accessed in a different way, and you can dramatically change the way your site behaves when you have crazy promotions.

When 800.com started our promotion, our sites dramatically switched from 95% of people browsing and 5% of people checking out, to 95% percent of people checking out and only 5% browsing. When the sites fundamental ratio changed we had to change the site.  We hadn't designed for this!

For the short term we started by turn the product catalog pages that were the most frequently visited into static pages. Don't go poo-poo that power of the static page. For small companies with just a few web servers there's little faster than serving a static page.

Cache Everything (Resources and Reference Data)

Memory is exceedingly cheap, and you really can't have too much.  Additionally if you're a small company with a small site, chances are your product catalog isn't that large. I would be surprised if most medium sized, catalogs can be kept in memory.  Additionally, the data in a product catalog using change very often, so it can be cached certainly for hours at a time, if not longer. After our short-term static files solution, we moved to just caching the entire catalog in memory. When all you need to do is show products and categories, caching is your friend.

Move Images Somewhere Else (Make it someone else's job)

We learned early on that the pages that the web servers that were doing the work really didn't need to be worrying about serving images, so we put those on http://images.800.com. This might seem obvious now in 2010, but it was pretty cool thinking in 1997. This allowed us to put all of our product images on a separate server farm and scale them out differently.

Today a lot of people put their images up on S3 at Amazon or other cloud services. For image heavy sites like product catalogs, moving them not only saves you time, effort and hardware, but it can also save bandwidth.

Partition Your Responsibilities (Break it up)

Even though you might have a single web server, or you might have six, think about partitioning responsibility early on, with separate mini-sites for things like images, profiles, web services, product catalogs, shopping cart, and check-out. If your site's characteristics change quickly and and you have to scale out and add new web servers, you can even create separate web farms for these mini-sites just checking out or just images. This can be as easy as having a separate virtual directory, and treating it as an application.

Embrace Stale Data (Realtime is a Lie)

One of the things we had at 800.com that was kind of revolutionary at that time was an AS/400 inventory system that we hooked up directly to the web.  You could see a real time inventory of any product even to the point where you could hit refresh and see products being sold. Remember that this was 1997.  In this case, it was silly of us to plug the AS/400 directly into the web server, so eventually we built an intermediate database. But the real root issue was that we realized what "realtime" meant to our business. The president of the company said he wanted realtime inventory, so we assumed he meant it.  But that didn't mean we needed to go back to the inventory system every second.  In fact, when pressed, we learned that realtime to our president could be within 5 or 10 minutes.

Updating data every 10 minutes is infinitely easier than updating it in real time as you might guess. Ask yourself, if you've got realtime data on an otherwise mostly static page, how stale can my data be?

Thanks for indulging me on this trip down memory lane.  Certainly, a lot of this stuff is obvious in 2010, and not just obvious but required in large enterprise systems, but these basic principles still apply today for small businesses running a relatively small web sites on, say, less than 10 servers.

Congratulations to SparkFun for their successful promotion. Even though I was unable to visit their site for two hours, the buzz they generated on twitter is no doubt invaluable.