Scott Hanselman

The Internet is not a black box. Look inside.

November 12, '12 Comments [31] Posted in Back to Basics | Musings
Sponsored By

All too often I see programmers trying to solve their problems on the internet by blindly "flipping switches."

Change something, hit refresh in the browser. "Why is that cached? What's going on?" Change something else, hit refresh in the browser. "What's the deal?"

You may have heard the term "cargo cult programming" where islanders after World War II would wave sticks hoping that planes full of supplies would fly over. They drew a conclusion that the sticks waving caused the planes to come.

Think about abstractions. This is a good reminder for the beginner and the long-time expert. This applies not just to computers but to cars, light bulbs, refrigerators and more.

What are you not seeing? Look underneath.

When coding on the web, remember that effectively NOTHING is hidden from you.

A friend emailed with a question about some CSS files not caching. This is a smart guy with a long question about a confusing behavior in the browser. I asked - as I often ask - what's happening underneath? Did you look inside?

Are you using Fiddler? Did you press F12 in your browser of choice and explore their network tools? Are you using WireShark?

Literally this moment, as I am writing this post, I just noticed that the Twitter box on my blog here doesn't have my latest tweet embedded.

Where's my tweet?

I could hit refresh a bunch of times, google around for vague terms, email a friend, or I could look inside.

I hit F12 in my browser. I look at the Network tab, and sort by Status.

Remember to use the Network Tools in your browser

Hey, suddenly my Twitter API call is a 404. First, that's lame of them. They should have redirected me, but alas, no one respects the permalink anymore. #getoffmylawn

With this single  insight I am now armed with googleable terms. I do a single search for "twitter user timeline json api" and see at the Twitter Developer Center that they've changed the format to included "api." and a version number.

I change my template to call this changed URL https://api.twitter.com/1/statuses/user_timeline/shanselman.json?callback=twitterCallback2&count=10 instead, and hit Refresh in my browser, once.

There's my tweet?

There's my tweet. No joke, this just happened. Good timing, I think.

You decide how deep you want to go down the rabbit hole. I am not expecting everyone to be a neurosurgeon or a professional network engineer but I firmly believe that digging just one layer deeper in all things will enhance your life and your work.

Learn basic HTTP debugging and ALWAYS check your result codes. Even if you are a non-technical blogger, learn how to check for 404s and 301s and 500s and assert your assumptions.

The world - and the internet - is not a black box. Look inside.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Monday, November 12, 2012 1:36:30 AM UTC
Yes, fiddling with at least one layer of abstraction down will at the very least give you the information you need to ask a specific question. At best, you could solve the problem yourself.
Monday, November 12, 2012 1:43:06 AM UTC
Never heard of that World War II thing, but from brief reading it sounds like the stuff B.F. Skinner was doing, Pigeon Superstition

http://www.youtube.com/watch?v=TtfQlkGwE2U

Or the description from the movie Mr Nobody:

http://www.youtube.com/watch?v=8uPmeWiFTIw
Monday, November 12, 2012 1:48:00 AM UTC
It seems like this is a fundamental requirement for good debugging of web stuff. When in doubt look under the hood.

I'm curious if this type of behavior is the result of certain styles of learning or teaching, or if we all do this at one point or another in our lives and its just a question of development/debugging maturity.
Steve Syfuhs
Monday, November 12, 2012 3:09:04 AM UTC
Can't help but think of this AI koan, via the Jargon File: Tom Knight and the Lisp Machine

A novice was trying to fix a broken Lisp machine by turning the power off and on.

Knight, seeing what the student was doing, spoke sternly: “You cannot fix a machine by just power-cycling it with no understanding of what is going wrong.”

Knight turned the machine off and on.

The machine worked.

Monday, November 12, 2012 3:12:49 AM UTC
Great post. It reminded me of your 'Please Learn to Think about Abstractions' post (http://www.hanselman.com/blog/PleaseLearnToThinkAboutAbstractions.aspx) with your wife's understanding of plumbing versus yours. Just digging that little bit deeper can make a world of difference.
Michael
Monday, November 12, 2012 3:40:16 AM UTC
I am a little shocked and dismayed at the level of "comfort" so-called web developers are these days. When I ask if they're looking at the raw output of the page (View source) or a real-time interpretation (Firebug) they don't know the difference. When I ask them to clear their cache I see them stumble around inside of Internet Explorer options instead of hitting F12 and using the tools there. When I talk about looking at the output sequence and mention Fiddler, they look at me like I have three heads.

I literally helped a team last week in two minutes when they had been looking at web.config files, IIS settings, and all kinds of mumbo-jumbo when I noticed they just had a mismatched tag in their output (which I saw immediately with a F12 inspect of the HTML because the tree structure wasn't right. This is neither rocket nor science yet developers blow things out of proportion and lead you down rat holes that you don't have to. Occam's Razor.

I think there's two important lessons these days that every web developer needs to know. This one about looking under the hood, and that not everything is seemingly as complex as one might take it to be.
Monday, November 12, 2012 4:03:59 AM UTC
I, too, am shocked at how few people know about this. F12 is the greatest key on browsers, and looking just one layer deep can tell you so much more about a problem.

I work at a company where the primary product is a web application, and it's amazing how many of the most experienced developers that work there will come to me and ask me what's going on. Depending on the nature of it, hitting F12 and looking at what failed to load, or what was cached, or even seeing the JS error can be found and give all the details that we need to pinpoint the problem.

Next time you have a problem with your website, hit F12 and take a look inside before asking someone else!

Good post Scott!
Richard Rout
Monday, November 12, 2012 6:06:09 AM UTC
Thanks Scott. Your post reminds me to my colleague that always ask about the problem on our commercial site. I just hit F12, look around and tell him what's the problem.

After that day, another guy come to me and ask the same question. Umm should I be a teacher?

This should be a basic idea for the web developer. I think.

Chatchawan chukorn

Chatchawan Chukorn
Monday, November 12, 2012 7:48:08 AM UTC
I agree, i seems to forget that often myself. Thanks for the reminder.
Monday, November 12, 2012 8:27:16 AM UTC
But, Scott... O_o that screenshot is not IE development tools! ;)

Just kidding, though I was a bit annoyed how almost everyone during //Build/ clearly forcibly used IE.

Anyway, on to the topic: you cannot imagine how many times I've gotten the inquiring looks from my colleagues - developers or even senior devs and/or architets - when I would ask about this same thing: "have you looked at the HTTP traffic? What does Fiddler say?"

8 times out of 10 they wouldn't even know what Fiddler is and 10 out 10 they would wonder why the hell would anyone look at HTTP traffic.

Keep up the good fight, the point about knowing the tools and the platform cannot be reinforced enough it seems (for the "kids these days"). Thanks.
Drazen Dotlic
Monday, November 12, 2012 9:14:18 AM UTC
I'm just glad Scott is using the alt property for images. Too few actually do.
DotNetDude
Monday, November 12, 2012 9:30:36 AM UTC
Certain countries block Twitter. You homepage keeps loading https://api.twitter.com/1/statuses/user_timeline/shanselman.json?callback=twitterCallback2&count=10 until it's timeout. It's absolutely not your issue, but it's a case
Monday, November 12, 2012 1:53:01 PM UTC
All this triage, tracking, fault-finding, analysis and bug-fixing is overrated. Personally, I've never yet found a problem I haven't been able to solve by blaming Microsoft.

;)
Monday, November 12, 2012 5:05:14 PM UTC
@Mark

I used to do that, until I realised that I've not once, in hundreds of dumps, encountered a crash caused by a Microsoft product.
K
Monday, November 12, 2012 6:15:24 PM UTC
I have to disagree with the title of your latest blog post. Black boxes are meant to be looked into.
Monday, November 12, 2012 6:22:41 PM UTC
I'm big on evangelizing both F12 tools (particularly Chrome's) and Fiddler, so many problems can be figured out by taking a look under the hood. Whenever I find a problem in other tech areas where I can't figure out some level of instrumentation to give me insight into the nature of the problem, I get frustrated much faster.


The thing is (and I'm not excusing it) - people often don't know that these tools exist because they've been doing the "same old thing" they've known their whole (development) life and haven't realized there is now a better way than CTRL+F5 -> View Source. Unfortunately, those types of developers are also less likely to see this post in the first place. If you help solve another developer's issue with Fiddler or another tool, please make a point of showing them what tools you used. Teach a man to fish and all of that. :)
Michael
Monday, November 12, 2012 6:22:43 PM UTC
This no longer comes as any surprise to me. Just look at Experts Exchange. The site is full of people that clearly don't have even a basic grasp of how the web works. Questions such as, "How can I print to the user's local printer from my code behind?" are frequent. Is it any wonder that the idea of inspecting the HTTP traffic scares the pants off these people. And yet they somehow keep getting jobs in the field. That is what truly baffles me.
Craig
Monday, November 12, 2012 7:27:33 PM UTC
Scott,

Great post. Too many call themselves 'web developers' and know nothing about HTTP. If you don't know at least something about the medium you're building in, you're bound to fall down hard during debug.

It's one of the reasons I'm so appreciative of MVC over Web Forms. Don't try to pretend HTTP is stateful, it isn't. You have several ways to send key/value pairs back to the server...that's it...so learn to love it!

Tony
Monday, November 12, 2012 8:02:24 PM UTC
DotNetDude - Thanks, it's for my blind readers.
Monday, November 12, 2012 8:43:11 PM UTC
I often use Fiddler and it has always provided enough over-the-wire information for me. What would you use WireShark for? Non-http data?

Seems like developers that started programming because they like to see how things work, tend to use these tools and get down to the lower layers. They are not intimidated by raw data and can find their way around it.


Richard
Richard Lemaster
Monday, November 12, 2012 8:54:37 PM UTC
The thing what's intriguing: why were you promoting a post from half a year ago?
Doeke
Monday, November 12, 2012 9:19:15 PM UTC
Nothing to do with monitoring network traffic (been doing that for *years* via NetMon, Fiddler, et al), but just a short recommendation about cargo cults.

Larry Niven's Dream Park is a { Mystery, Science Fiction, Gaming, Fantasy, All of the above} novel, set in a cargo cult veneer. Good read.
Larry Smith
Monday, November 12, 2012 9:46:21 PM UTC
Doeke? Huh? This is a new post.
Scott Hanselman
Monday, November 12, 2012 9:59:11 PM UTC
Larry Smith - Anything by Larry Niven (ok, almost anything) is a top-notch read!

Scott - great post. In a backwards way, it is encouraging to know that I am not the only one who isn't sure where to start looking when something goes wrong on the web (I'm slowly trying to move from Winforms/work-related dull stuff to new, exciting, and forward-moving web stuff).

I did, however, recently grab a book on HTTP, and decided to set up a Linux machine to supplement my MS-centric web learning. With my Windows/VS box, it's too easy to miss out learning the fundamentals, because there is all kinds of magical help designed to make your life easier. This is great for pros who know how to do it, and want to save some time (or beginners just getting started).

The Linux box will not have all the familiar assistance I've become spoiled with on the MS side (Gonna try to do this with as much Bash shell as possible), and I will therefore be forced to learn it the hard way.

THEN I can use the helpful tools (on either platform) which provide the comfortable abstractions.
Tuesday, November 13, 2012 11:53:25 AM UTC
Scott

Great post, I 100% agree.

I partially also blame tools like Visual Studio for this ignorance. Sometime important concepts are hidden from the developer.
Leaky abstractions like webforms don't help either, methinks. There is no state in HTTP.

Just my 2 cents.
Kay
Tuesday, November 13, 2012 5:08:02 PM UTC
Right after I read this post, a friend called me asking me to help him figure out why an external javascript wasn't loading... I almost sent him a link to this page.
Ryan
Wednesday, November 14, 2012 12:49:50 AM UTC
Well actually, without no much knowledge and good tools you can deduce lots of thing, I wrote an article how I did to attach an azure blob drive on windows 7. The point of the article was not about azure, but about how to look inside, and use it at you advantage.
Nicolas D.
Wednesday, November 14, 2012 10:12:24 AM UTC
We have a new web developers and IE wasn't putting transparency on a div. I didn't know about this bug and I asked him why he put the transparent background image on there. His response was "because it said so on Stack Exchange".
Wednesday, November 14, 2012 3:37:28 PM UTC
Sometimes even on your computer you see these weird codes such as unidentified computer accessing network makes me think something sinister is going on.
Thursday, November 15, 2012 7:24:08 AM UTC
Great Post scott ! I always read each and every comments on each of your post, they are useful, educating and few like Doeke's are entertaining too.
Friday, January 04, 2013 12:08:00 AM UTC
Scott... I really respect you as a developer, but I must say you're completely wrong on this one. The internet is DEFINATELY a black box. This Scott is the internet http://www.youtube.com/watch?v=sDA1HUmuuJo
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.