Scott Hanselman

Back To Basics: Algorithms and Going Back To Virtual School

September 17, '08 Comments [18] Posted in Back to Basics | Learning .NET
Sponsored By

imageBefore I graduated from College/University I was convinced that school was for lamers. Then I graduated from school and decided that NOT going to school was for lamers. That shows you what a wishy-washy person *I* am. ;) School is for some folks and not for others. Wear the shoe that fits you best.

Totally random retrospective self-focused aside: I graduated from OIT with a BS in Software Engineering in 2003. Yes, that's 2003. It took me only 11 years to get a 4-year degree. ;)
I've been a programmer since 1992, so about 16 years, but 11 of those I was going to school at night. Basically I worked 9a-5p and went to school 6p-10p. For the first few years I took a lot of courses, but then work happened and I tapered off to one course a term. This was fine for a long time but then credits started falling off the other side. Basically courses I took 7+ years before became obsolete. I actually lived through four Deans while I was there. I made a deal with one of them that if I taught .NET I could keep my credits longer. That let me keep taking courses while working the whole time. The result took 11 years and two schools. But now I can just say, "I've got my degree and just not mention that I got it 5 years ago. Don't tell. Lots of experience sounds better than "slow learner at school."

Whether you went to school or are self-taught, even though the Internet was originally more a place to put academic papers than a place to meet "friends", there's a LOT more really good academic information on the web than I remember. It also takes more interesting forms than just pages of class syllabi. Here's the syllabus for the CST407 class I taught in Fall of 2003.

You can learn via Googling, you can learn via Book Readin' but you can also setup a more focused "curriculum" for yourself. I know a lot of devs that I admire that do this. They'll pick a discipline, area, language, whatever, and create a mini-class for themselves. Actually that much-teased "Learn Whatever in 24 Hours" books have more structure in this way than most programming books.

These days there's all the good content at iTunesU from colleges that didn't return my calls back in my High School years. There's even 1986 video of the legendary MIT lectures "Structure and Interpretation of Computer Programs" with Hal Abelson and Gerald Jay Sussman. There's MIT's excellent OpenCourseWare Videos of the Introduction to Algorithms class up on Google Video.

Professors are creating better and better tools to visualize things, like this amazing JavaScript-based page on "Animated Sorting Algorithms" by David R. Martin. It's utterly brilliant, not just because it's a Visualizer, but because the code is in Javascript, so you're seeing it happen. It's impressive like when you're playing a video game and you discover a particularly amazing cut-scene isn't prerendered, it's rendered in-engine. (UPDATE: And it would be impressive if it were true. I'm crushed. They ARE prerendered. Well, still. It's cool. Thanks, Adam!)

I recently stumbled on Massimo Di Pierro's site and his code. He's got a CSC309/321 class that he teaches at DePaul University and has a great Python application called "Algorithms Animator." His slides on basic OOP in C++ are excellent but the Algorithms Animator is really cool. There's a video of Algorithms Animator running over at Vimeo.

He's got all sorts of common algorithms, sorts, traversals, searches, etc. The algorithms are all in src/csc321algorithms.py, like this simple binary tree example.

def isNullTree(tree):
if tree is None: return true
if len(tree)==0: return true
return false

rootnode=0
leftchild=1
rightchild=2

def BinaryTree(node,left=[],right=[]):
list=[node,left,right]
if not isNullTree(left):
left[rootnode].parent=list
if not isNullTree(right):
right[rootnode].parent=list
return list

There's also more complex ones like a Huffman encoding example. But it's not the source code that interesting, although it is. The syllabus is really detailed, with the kind of detail you really only see in academia:

Huffman Encoding Definition: A minimal variable-length character encoding based on the frequency of each character. First, each character becomes a trivial tree, with the character as the only node. The character's frequency is the tree's frequency. The two trees with the least frequencies are joined with a new root which is assigned the sum of their frequencies. This is repeated until all characters are in one tree. One code bit represents each level. Thus more frequent characters are near the root and are encoded with few bits, and rare characters are far from the root and are encoded with many bits.

The Program accepts the text to be compresses as input and produces a text report showing compression rules and compressed text. The Program also shows in an animation how the Huffman tree is built.

Huffman encoding provides an example of Greedy strategy.

What's cool about this guy's way of teaching algorithms is that his app uses animation to show the algorithm happening step by step.

For example, if I use Huffman Encoding on the string "Scott Hanselman," here's Frame 0 of the animation.

"First, each character becomes a trivial tree, with the character as the only node."

Huffman Encoding Tree

Here's Frame 15 as the the character's frequencies are counted and the tree is in the process of being built.

"One code bit represents each level. Thus more frequent characters are near the root and are encoded with few bits, and rare characters are far from the root and are encoded with many bits."

Huffman Encoding Tree (2)

Here's Frame 22 as the tree has been built out and the bit values are applied.

Huffman Encoding Tree (3)

...and the result is in the image below showing the compression rules/map and the final bits.

image

I'm not sure about you, but I'm not able to bust out a QuickSort on a whiteboard under pressure anymore (I'm old).  I found this guy's stuff to be a great algorithms refresher, and using the little Python app to explore them made it very enjoyable. I wish we had this kind of stuff when I was in school. It's kind of nice to pretend I'm back in an Algorithms class and tickle the neurons that haven't fired in a while.

Related Posts

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Baby Sign Language 2.0

September 16, '08 Comments [27] Posted in Musings
Sponsored By

UPDATE: Check out http://www.babysignlanguage.com for more info on Babies and Sign Language!

The "2.0" in the post title there refers to Baby #2 who is pushing 9 months now. Insert gratuitous baby footage here (It's all good, but the best bit is just after 1:23):

He's a clever little dude, I like to think. One of the things that we did with Baby #1 (now Toddler of Pure Evil©) was that we taught him Baby Sign Language. I LOVE talking to parents about the benefits of teaching their kids a foreign language and American Sign Language is a full fledged foreign language. It just happens to be one that babies can learn before they can speak.

Teaching the first boy Sign was a radical success for our little family, so we're starting it up with 2.0. We're teaching him American Sign Language.

When?

How much earlier? Anywhere from 6 months old to one year, a baby can clearly make their intent known with Sign. Now, I'm not talking about trying to push a baby to be "gifted" or to get a baby to do anything that isn't relaxed and natural. I'm not promoting trying to teach 2 year olds to take college entrance exams.

Why?

What I am a huge fan of is taking the communication you've already got with your baby* and elevating the grunts and points that are primitive, invented signs, and replacing them with a formalized system of signs that have been used for years by deaf folks.

*(in the range between 6 months and whenever they start really talking...between 18 mos and 2 years or whatever)

With Baby Sign Language you start by introducing "Needs-Based" Signs like Food, Milk, Mommy and start using them all the time. Every time you say food out loud, you also sign food. Then you start introducing repetitive action signs. Those are signs for stuff that happen all the time like Sleep or seeing Grandpa.

What a Weird Country

A lot of folks I meet while (or from) overseas think this is insane. A German woman told me once "I know exactly what my baby wants." And that's cool. However, sometimes it's nice to hear directly from the child.

The idea is that the main reason babies cry is that their needs aren't being met. I'm hungry. I'm tired. I want that ball. We spend two years or more with these babies pointing and grunting at stuff trying to get their point across.

Also, "I'm hungry" is pretty vague. It's a joy to see a 14 month old sign "Want Grapes" then calm down when they get exactly what they want. It might sound weird, but it can't hurt to try! It's becoming more and more popular in the States and it's because it works and it enhances my relationship with my kids very early on.

How Much Sign Can They Learn?

It's a complete language and many successful Deaf folks would say "all of it." They'll learn as much as you can throw at them. However, I suggest picking 3-5 signs from about 6 months old until they've mastered them. Then, start adding a sign a week. Then a sign a day. Then, once you realize your child is smarter quicker to absorb then you are, they're probably going to start speaking soon anyway.

Boy #1 ended up with something like 80 signs (lots of Animals) and then just started talking one day.

Won't It Slow Their Development?

In my experience, nope. If anything, it gives them a nice little bump ahead, but that's not the point of doing it. Our two year old is speaking nicely and has happily forgotten his signs as soon as he figured it that speaking was easier for him. Now, fast forward a bit, he's re-learning it and signing (and speaking) to his little brother.

Check out my (now series of) posts on Baby Sign Language:

I highly recommend the Baby Signing Time DVDs and CDs. Now, forgive me as I compile a few small summaries from those previous posts:

What do you need to do to start signing?

  • Check your local community center. They often offer Baby Sign Language classes. We took classes before Z was born, and when he was 6 months old.
  • If Baby Sign Language is unusual or unused in your country, either find some Deaf Folks and learn your country's specific Sign Language, or use ASL (American Sign Language). The trick is to be consistent and have an illustrated dictionary to refer to.
  • Stick with it. Don't give up. We started when he was six months old and signed every day without a single clear response until he was a year old. We nearly quit a dozen times before that.
    • Then one day he signed "light" as clear as day in his bedroom. We turned on the light and our sonlit up with a small as wide as his face. That's when we connected with him. I'm not talking about the standard Mom/Dad/Baby we-love-you connection. I'm talking about the baby's opinion matters kind of connection.
  • Get picture books, lots of them, and learn the signs for the animals. I highly recommend the Priddy Books series of books for baby.
    • Learn the signs for animals and common objects and use them every time you see one out in the world. We went for a walk on the Portland Waterfront today and our son was signing bird anddog and plane and sharing those discoveries with us. It's great when he sees something interesting and points at it, but it's something different when he signs about something we didn't see.
  • Pay Attention and prepare for the unexpected.
    • Example: The baby was frantically signing ball recently, gesturing wildly at a dog. We tried to correct him..."No no sweetie, that's a dog, not a ball." The dog lifted it's head and we saw that the dog was in fact playing with a ball that we hadn't seen.

Testimonials

Many of my friends and family have taught their kids sign. For many, including all the non-Americans, they were teased by family and friends - especially concerned mother's-in-law. But they stuck with it. My friend Daniel "Kzu" Cazzulino had a great experience with Baby Sign Language in Argentina:

Just like Scott felt, it's not just a matter of teaching her something to make her "smarter" early on. There's a new kind of connection that you can make with your baby. Aylen's face shines when she sees that we can listen to her needs and help her. She no longer cries when she's hungry or thirsty, or when she wants to take a bath. That's huge."

Craig Andera is also huge Baby Signing Fan. He had to have patience early on though:

Just like Scott, it was initially like signing to a wall. She didn't seem to care, and she certainly didn't sign back. But I knew from my brother that it was just a matter of time, and sure enough, at about eight months, Ellen was able to mime the sign back to us. It's pretty amazing to get any communication whatsoever (other than smiling and crying) from an eight-month-old.

It's moving slowly, but our 9 month old (after flat ignoring our signs and smiling for the last 3 months straight) now asks for More Food quite clearly. I look forward to the next year that I get to "talk" to him as I patiently wait for him to actually start talking.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Hanselminutes Podcasts 128 and 129 - Ajax with Scott Cate and JavaScript with Bertrand Le Roy

September 16, '08 Comments [3] Posted in ASP.NET | Javascript | Podcast
Sponsored By

Ajax LogoMy one-hundred-and-twenty-eighth and one-hundred-and-twenty-ninth podcasts are up. Both are on JavaScript and each has a very different perspective. I also talked to jQuery creator John Resig a few weeks ago so check that out. Watch for a possible upcoming podcast with the creator of JavaScript, Brendan Eich, soonish. I'm trying to give JavaScript pretty full coverage. If anyone wants to introduce me to Douglas Crockford, please do so. ;)

Anyway, the first show is "Object Oriented AJAX with Scott Cate." Surprisingly, to me, a number of folks didn't agree with Scott's perspective and some even felt he "got it all wrong." However, Scott's go a very successful, very Javascript heavy product, so he must be doing something right. Scott's into MS-AJAX and is even a fan of the UpdatePanel. Apparently at this point some listeners cried. I love a pragmatist.

The second show is "Thoughts on Javascript with Bertrand Le Roy." Bertrand is the PM for MS-AJAX, and a JavaScript Ninja. We chat about where he thinks Javascript is going and how fast it'll really get. He also re-explains JavaScript's prototype model and how OOP works in a JS world. We finish up with a good understanding on how BLR's new JavaScript Templating system (preview) will let me write more Ajax-using sites with less JavaScript and more declarative HTML.

Subscribe: Subscribe to Hanselminutes Subscribe to my Podcast in iTunes

Do also remember the complete archives are always up and they have PDF Transcripts, a little known feature that show up a few weeks after each show.

Telerik is our sponsor for this show.

Telerik's new stuff is pretty sweet, check out the ONLINE DEMO of their new ASP.NET AJAX suite. RadGrid handles sorting, filtering, and paging of hundreds of thousands of records in milliseconds, and the RadEditor loads up to 4 times faster and the navigation controls now support binding to web services on the client.

As I've said before this show comes to you with the audio expertise and stewardship of Carl Franklin. The name comes from Travis Illig, but the goal of the show is simple. Avoid wasting the listener's time. (and make the commute less boring)

Enjoy. Who knows what'll happen in the next show?

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

The Weekly Source Code 33 - Microsoft Open Source inside Google Chrome

September 12, '08 Comments [42] Posted in Microsoft | Programming | Source Code | Windows Client
Sponsored By

First, let me remind you that in my new ongoing quest to read source code to be a better developer, Dear Reader, I present to you thirty-third in a infinite number of posts of "The Weekly Source Code."

That said, what does Microsoft Code have to do with Google Chrome, the new browser from Google? Take a look at the Terms and Conditions for the "Chromium" project up on Google Code. There are 24 different bits of third party software involved in making Chrome work, and one of them is WTL, the Windows Template Library, which was released as Open Source in 2004.

Chrome's use of the Open Source Windows Template Library

WTL is distributed under the MS-PL or Microsoft Public License. This is a VERY relaxed license that basically says "have fun, and don't call if there's trouble." In the Open Source world, licenses like that make people smile.

WTL is a C++ library for Win32 development and is kind of like MFC (Microsoft Foundation Classes), but a lot more lightweight. It was originally worked on by Nenad Stefanovic as an internal thing at Microsoft that was then released as an unsupported sample. Nenad is still listed on the old SourceForge project.

WTL embraces ATL (Active Template Library) which is a series of C++ template classes made to make COM development easier. It was more or less patterned after the STL (Standard Template Library). You may remember that folks like Chris Sells were hard-core ATL wonks. Chris and Chris Tavares (of P&P fame) worked on the ATL Internals book.

WTL was well loved going back to 2002. There's a great post from back then by Simon Steele about The Joys of WTL. Simon says:

The Windows Template Library (WTL) is MFC on template-based steroids - after a successful stretch on the slimfast plan. WTL provides the user-interface frameworks that we need to write proper GUI applications without resorting to the bloated MFC or complicated pure Win32 API programming. A number of the "must-have" classes from MFC are also available as WTL utility classes too, welcome back your old friends CPoint, CSize, CRect and most importantly CString! WTL produces small executables that do not require the MFC run time libraries - in fact if you stay clear of the Visual C++ runtime functions (strcpy and friends) you can dispense with msvcrt.dll as well - leading to really small programs, which run fast too

Windows Template Library was released as Open Source over 4 years ago which is like 28 Internet years. May of 2004 was a long time. I didn't think Microsoft was doing much Open Source that far back, but it appears they were. In fact, back as far as April of 2003 there was talk on the WTL Yahoo Group by Pranish Kumar, then of the Visual C++ team, of trying to figure out how to get the product out into the community in a way that would let it live on.

History: How WTL Became Open Source

I had an IM chat today with Pranish Kumar about how WTL made it out of a 2004 Microsoft as an Open Source project. I'd also encourage you to check out both the Shared Source page at MSFT, the Open Source page, and most importantly, Port 25.

Here's part of my IM with Pranish about how WTL was released:

[WTL] was one of the first if not the first OSS things from Microsoft and it was a tough sell. There was a meeting with some bosses where we were presenting 3 potential OSS items. I guess it was the first "real OSS" with joint MS/Community involvement as opposed to just us posting something externally. WTL was the only one that got approved.

Me: Did it start the Shared Source Initiative?

Yes in the broader sense, I think we took the basis for the license/process from Win CE and a few other components which Microsoft made available (in some form) as shared source. They also looked at BSD and some other licenses.

It was a fascinating experience for many reasons. One of them was seeing the reaction of various Microsoft execs to the whole open source/shared source idea. There was a lot of concern about OSS = Linux, and questions on whether there was business value in us engaging

It's pretty amazing how our stance/attitude has changed, one of the reasons WTL got through is because we convinced management, it had a passionate community base and would really help us foster that base.

I check in on the community now and then (not as regularly as I'd like) and I'm always impressed how it's remained strong.

One of the reasons I wanted to work for ScottGu was because of Microsoft's always improving attitude about releasing source. It's a big company and sometimes moves slow, but more people "get it" now than before.

Digging In

Chrome uses abstraction libraries to draw the GUI on other non-Windows platforms, but for now, what sits underneath part of ChromeViews is good ol' WTL. Makes sense, too. Why not use a native library to get native speeds? They are using WTL 8.0 build 7161 from what I can see.

Chromium is a lot of code. The source tarball is over 400 megs, if you want to try to compile it yourself with VS2005. Let's try to look at a few tiny interesting bits, though. You can check out their "Build Bot" if you like, and watch the development on the Linux and Mac Versions as they progress each day.

In some places, Chrome uses WTL for little stuff, like macros. For example, in the Chrome AeroTooltipManager, GET_X_LPARAM is a macro:

...snip...
if (u_msg == WM_MOUSEMOVE || u_msg == WM_NCMOUSEMOVE) {
int x = GET_X_LPARAM(l_param);
int y = GET_Y_LPARAM(l_param);
if (last_mouse_x_ != x || last_mouse_y_ != y) {
last_mouse_x_ = x;
last_mouse_y_ = y;
HideKeyboardTooltip();
UpdateTooltip(x, y);
}
...snip...

In other places, they rely on it more, like in text_field.cc that includes atlcrack.h. These are not drugs, mind you, but rather "message crackers" to help get at, and react to, the information inside Window Messages. These are used to create a "message map" of all the events you're interested in. These are macros that expand into an obscene amount of code. They are exceedingly handy.

// CWindowImpl
BEGIN_MSG_MAP(Edit)
MSG_WM_CHAR(OnChar)
MSG_WM_CONTEXTMENU(OnContextMenu)
MSG_WM_COPY(OnCopy)
MSG_WM_CUT(OnCut)
MESSAGE_HANDLER_EX(WM_IME_COMPOSITION, OnImeComposition)
MSG_WM_KEYDOWN(OnKeyDown)
MSG_WM_LBUTTONDBLCLK(OnLButtonDblClk)
MSG_WM_LBUTTONDOWN(OnLButtonDown)
MSG_WM_LBUTTONUP(OnLButtonUp)
MSG_WM_MBUTTONDOWN(OnNonLButtonDown)
MSG_WM_MOUSEMOVE(OnMouseMove)
MSG_WM_MOUSELEAVE(OnMouseLeave)
MSG_WM_NCCALCSIZE(OnNCCalcSize)
MSG_WM_NCPAINT(OnNCPaint)
MSG_WM_RBUTTONDOWN(OnNonLButtonDown)
MSG_WM_PASTE(OnPaste)
MSG_WM_SYSCHAR(OnSysChar) // WM_SYSxxx == WM_xxx with ALT down
MSG_WM_SYSKEYDOWN(OnKeyDown)
END_MSG_MAP()

They also use some handy helpers that are C++ classes around Windows structures. For example, the Windows POINT structure is a class in WTL called CPoint. The class actual derives from the struct. Lots of interesting stuff in there, and WTL is at a pretty low level helping out and keeping things tidy.

Now, moving on to something I found fascinating because it's not documented and may or may not have required some disassembling to accomplish.

Chrome's Odd Use of Data Execution Prevention

This part isn't explicitly about use of open source, but it's darned interesting. This is part of Chrome's WinMain(). It's long, but check out a few interesting bits. First, the big if/else at the beginning. They look at the command line and determine if they (the EXE) are one of three flavors...either a Renderer, a Plugin [host] process, or the Browser process. Notice that they have DEP (Data Execution Prevention) turned on for the Renderer and main Browser, but have to enable ATL7 thinking because there are plugins that weird build in older ways still out there. They are ultimately calling SetProcessDEPPolicy and passing in a flag to enable DEP, as well enabling ATL7 compiled processes. From MSDN help:

"Disables DEP-ATL thunk emulation for the current process, which prevents the system from intercepting NX faults that originate from the Active Template Library (ATL) thunk layer."

These new APIs were added in Vista SP1, Windows XP SP3 and WIndows 2008. Why is ATL special cased? From Michael Howard:

"Older versions of ATL, and by older I mean pre-Visual C++ 2005, used dynamically generated code in small isolated cases. Obviously, without the appropriate APIs this is going to cause problems on a DEP-enabled computer, because you can't execute data. This code is referred to as a "thunk" and versions of ATL in VC++ 2005 and later work correctly with DEP."

Some plugins that might run in a Chrome sandboxed process might be compiled in this way, so that process has a different security DEP setting than the others.

int APIENTRY wWinMain(HINSTANCE instance, HINSTANCE prev_instance,
wchar_t* command_line, int show_command) {
// The exit manager is in charge of calling the dtors of singletons.
base::AtExitManager exit_manager;

// Note that std::wstring and CommandLine got linked anyway because of
// breakpad.
CommandLine parsed_command_line;
std::wstring process_type =
parsed_command_line.GetSwitchValue(switches::kProcessType);

const wchar_t* dll_name = L"chrome.dll";
if (process_type == switches::kPluginProcess) {
// Plugin process.
// For plugins, we enable ATL7 thunking support because we saw old activex
// built with VC2002 in the wild still being used.
sandbox::SetCurrentProcessDEP(sandbox::DEP_ENABLED_ATL7_COMPAT);
} else if (process_type == switches::kRendererProcess) {
// Renderer process.
// For the processes we control, we enforce strong DEP support.
sandbox::SetCurrentProcessDEP(sandbox::DEP_ENABLED);
} else {
// Browser process.
// For the processes we control, we enforce strong DEP support.
sandbox::SetCurrentProcessDEP(sandbox::DEP_ENABLED);
}
...snip...
}

When you dig into their use of DEP, notice this interesting comment, as they try to get DEP working under Windows XP SP2 and Windows Server 2003 SP1. They are using the totally unsupported technique outlined in this article from 2005 to try to turn on DEP. If you try to call this on Vista you'll get back STATUS_NOT_SUPPORTED, of course. ;) There's an official Vista API, and that's SetProcessDEPPolicy.

As an side, and interestingly enough, this undocumented API has been added as a patch just last week to WINE (Windows Emulation) for those who try to emulate Windows under Linux, but outside a VM.

Note the most interesting comment in the method:

"// Completely undocumented from Microsoft. You can find this information by
// disassembling Vista's SP1 kernel32.dll with your favorite disassembler.
enum PROCESS_INFORMATION_CLASS {
ProcessExecuteFlags = 0x22,
}"

Looks like The Chromium authors may have disassembled part of the Windows Kernel in order to achieve this security feature under Windows XP SP2. Probably not cool to do that, but they're clearly doing it for good and not evil, as their intent (from reading their code) is to make their browser safer under XP SP2 and prevent unwanted code execution.

This internal and totally unsupported API is in the Microsoft Windows Internals 4th Edition, Chapter 6, on download.microsoft.com (PDF). It's also mentioned in a Microsoft Research PowerPoint (PPTX). An architect on the Windows Kernel team point out in a forum posting that this was internal:

"I want to stress as a disclaimer that NtSetInformationProcess, class ProcessAccessToken, is an undocumented and unsupported infterface. It is reserved for system component use and is subject to change between operating system releases"

You can see the dance Chrome does below or on their source site. They poke around looking for a method that does what they want, using GetProcAddress:

namespace sandbox {

namespace {

// These values are in the Windows 2008 SDK but not in the previous ones. Define
// the values here until we're sure everyone updated their SDK.
#ifndef PROCESS_DEP_ENABLE
#define PROCESS_DEP_ENABLE 0x00000001
#endif
#ifndef PROCESS_DEP_DISABLE_ATL_THUNK_EMULATION
#define PROCESS_DEP_DISABLE_ATL_THUNK_EMULATION 0x00000002
#endif

// SetProcessDEPPolicy is declared in the Windows 2008 SDK.
typedef BOOL (WINAPI *FnSetProcessDEPPolicy)(DWORD dwFlags);

// Completely undocumented from Microsoft. You can find this information by
// disassembling Vista's SP1 kernel32.dll with your favorite disassembler.
enum PROCESS_INFORMATION_CLASS {
ProcessExecuteFlags = 0x22,
};

// Flags named as per their usage.
const int MEM_EXECUTE_OPTION_ENABLE = 1;
const int MEM_EXECUTE_OPTION_DISABLE = 2;
const int MEM_EXECUTE_OPTION_ATL7_THUNK_EMULATION = 4;
const int MEM_EXECUTE_OPTION_PERMANENT = 8;

// Not exactly the right signature but that will suffice.
typedef HRESULT (WINAPI *FnNtSetInformationProcess)(
HANDLE ProcessHandle,
PROCESS_INFORMATION_CLASS ProcessInformationClass,
PVOID ProcessInformation,
ULONG ProcessInformationLength);

} // namespace

bool SetCurrentProcessDEP(DepEnforcement enforcement) {
#ifdef _WIN64
// DEP is always on in x64.
return enforcement != DEP_DISABLED;
#endif

// Try documented ways first.
// Only available on Vista SP1 and Windows 2008.
// http://msdn.microsoft.com/en-us/library/bb736299.aspx
FnSetProcessDEPPolicy SetProcDEP =
reinterpret_cast<FnSetProcessDEPPolicy>(
GetProcAddress(GetModuleHandle(L"kernel32.dll"),
"SetProcessDEPPolicy"));

if (SetProcDEP) {
ULONG dep_flags;
switch (enforcement) {
case DEP_DISABLED:
dep_flags = 0;
break;
case DEP_ENABLED:
dep_flags = PROCESS_DEP_ENABLE |
PROCESS_DEP_DISABLE_ATL_THUNK_EMULATION;
break;
case DEP_ENABLED_ATL7_COMPAT:
dep_flags = PROCESS_DEP_ENABLE;
break;
default:
NOTREACHED();
return false;
}
return 0 != SetProcDEP(dep_flags);
}

// Go in darker areas.
// Only available on Windows XP SP2 and Windows Server 2003 SP1.
// http://www.uninformed.org/?v=2&a=4
FnNtSetInformationProcess NtSetInformationProc =
reinterpret_cast<FnNtSetInformationProcess>(
GetProcAddress(GetModuleHandle(L"ntdll.dll"),
"NtSetInformationProcess"));

if (!NtSetInformationProc)
return false;

// Flags being used as per SetProcessDEPPolicy on Vista SP1.
ULONG dep_flags;
switch (enforcement) {
case DEP_DISABLED:
// 2
dep_flags = MEM_EXECUTE_OPTION_DISABLE;
break;
case DEP_ENABLED:
// 9
dep_flags = MEM_EXECUTE_OPTION_PERMANENT | MEM_EXECUTE_OPTION_ENABLE;
break;
case DEP_ENABLED_ATL7_COMPAT:
// 0xD
dep_flags = MEM_EXECUTE_OPTION_PERMANENT | MEM_EXECUTE_OPTION_ENABLE |
MEM_EXECUTE_OPTION_ATL7_THUNK_EMULATION;
break;
default:
NOTREACHED();
return false;
}

HRESULT status = NtSetInformationProc(GetCurrentProcess(),
ProcessExecuteFlags,
&dep_flags,
sizeof(dep_flags));
return SUCCEEDED(status);
}

} // namespace sandbox

It's a really interesting read and there's a lot of stuff going on in the comments, like TODOs, HACKs, and the like. All the stuff you'd expect to see any application of significant size. Funny, it's been at least 5 years since I've thought about C++ deeply. And to think I used to do all this -> stuff full time for money!

There's lots more to see. Check out the About Box version checks where they were blocking on Vista SP1 with UAC disabled. Also, the Threading stuff is interesting as they have a Thread Class that was ported to Mac and Linux. Finally check out Pickle.cc, as they serialize objects by "pickling them." Pickle is serialization for Python, and this looks like they're serializing between C++ and Python, and this is a C++ implementation of Pickle.

Back on WTL, you can download the final MS release of WTL 7.1 at Microsoft Downloads if you're interested. However, the more interesting release is the 8.0 release from June of 2007. This was the most recent release from the community! WTL 8 includes full support for Vista!

I think it's great that Microsoft is releasing more and more code in either Shared Source, Reference Source, or my favorite, Open Source as MS-PL. The fact that Google was able to use it, even this small part, really speaks to the spirit of Open Source.

Related Links

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

The .NET Framework and the Browser's UserAgent String

September 11, '08 Comments [42] Posted in ASP.NET | Programming
Sponsored By

Vote Now! One of the things I'd said I'd do, Dear Reader, when I went to work for the Big Blue Monster, was to get your feedback into The Company whenever possible. Here's another opportunity for us to effect change, however small.

A while back I made a little site called http://www.smallestdotnet.com that would look at your Browser's UserAgent

and let you know the size that the latest .NET Framework would be for you (ballpark size). In the process I've gotten to see a LOT of interesting Browser UserAgents in the IIS logs.

If you visit the site and scroll down, you'll see YOUR Browser UserAgent at the bottom. Here's mine:

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; WOW64; Trident/4.0; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.5.21022; Media Center PC 5.0; Zune 2.5; MS-RTC LM 8; .NET CLR 3.0.30618)

Notice that I've got IE8 on 64-bit Windows. However, there's other stuff in there, like I have the Zune software, I have Media Center enabled, and I've got three versions of the .NET Framework. In this example, 2.0, 3.0, and 3.5.

Here's a UserAgent that showed up today on http://www.smallestdotnet.com:

Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 1.0.3705; InfoPath.1; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 1.1.4322; .NET CLR 3.0.04506.590; .NET CLR 3.5.20706; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; Zune 2.5; WWTClient2; MS-RTC LM 8; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)

Now, this guy is clearly a developer who has installed beta versions of things, but he's also got InfoPath and a bunch of other stuff. Still, that's a pretty crazy UserAgent and it's being transmitted all the time.

Registry Editor

UserAgents are added for IE in the Registry, as seen in the screenshot above.

Why Should You Care?

In the case of a system with IE and a lot of .NET Framework versions, ASP.NET currently truncates the UserAgent at 256 characters so my own case, the site was only getting part of the story. They might change that truncation number, but ultimately, I think we all want tidy UserAgents, right?

UserAgent length matters. Early versions of FireFox had a UserAgent Limit of 127 characters and used to return null before the bug was fixed last month and now it truncates. The FireFox plugin added when you installed .NET by default only shows the most recent .NET Framework. Do you want that functionality in .NET?

Truncated UserAgents can also mean you might get "Unknown" Back from Request.Browser.Browser. In my case, I was getting an exception from Request.Browser.ClrVersion because the truncated CLR Version was something like ".NET CLR 3.".

There's some people looking at this internally, and there's three groups involved. There's Internet Explorer, involved peripherally, there's the CLR Team and the installer that adds the values, and there's the ASP.NET team who cares because of the server-side sniffing.

Poll/Questions

Vote Now!The questions I have for you are these:

  • Do you use these .NET Framework-specific values? What for? How do they improve your site or make your user experience different?
  • If the .NET Framework-specific values were removed completely what would that mean to you or your business?
  • What if the values were conflated to show just the most recent .NET Framework?
    • Could you get the information you needed using just the value of the latest version?
  • What if the values were conflated to show just the major side-by-side CLR releases?
    • For example, 1.x, 3.5SP1 and 4.0. In that example, 3.5SP1 implies 2.x and 3.0. If you just had 2.0, you'd see 1.x, 2.0, and that'd be it.
    • Could you get the information you needed using just the value of the those major CLR/Framework versions?

Please take a moment and do this micropoll. It's ONE question with ONE vote button. Literally two seconds of your time. You can also add the poll to your blog. Also, leave comments and I'll make sure the right people seem them.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.