Scott Hanselman

Abusing the Microsoft Research's Touch Mouse Sensor API SDK with a Console-based Heat-map

January 19, '12 Comments [24] Posted in Coding4Fun | Tools
Sponsored By

In August I purchased and reviewed the Microsoft Touch Mouse. I still use my Microsoft Arc Mouse more than the touch, initially due to what I felt was dodgy scrolling performance on the Touch Mouse, as I mentioned in my review. Still, I've kept it in my backpack and I use the Touch Mouse perhaps a few times a month and have kept the software up to date in case there's some software changes made to improve performance.

I can happily say that they've changed something and the scrolling performance is WAY better. I can finally get 1 to 2 pixel precision with it while scrolling in my browsers. The other nice feature is the "three finger swipe up" which gives you effectively a Windows version of the Mac Expose window switcher view.

Today I noticed while catching up on Long Zheng's excellent blog that the Touch Mouse Sensor SDK is available for download. Per their site:

"The Microsoft Touch Mouse supports multitouch gestures via an integrated sensor. The Touch Mouse Sensor API SDK is a small library intended to enable students and researchers to experiment directly with Touch Mouse sensor output.

Using this SDK, you can create your own applications consuming the 13×15 sensor image. The SDK includes C# and C++ samples demonstrating how to read and manipulate sensor data"

OK, so what can we do with this thing? It comes with a number of samples to get the bitmap from the sensor and process it. It includes examples in both C# and C++. Because I don't have C++ on this machine I got an error opening that solution, but you can just remove that project and still build the samples.

You don't need to do your work in a graphical application, although it's nice to visualize this data of course. First sample is a console app, that tells you were the center of mass of your big fat finger is.

The smarts are in the Microsoft.Research.TouchMouseSensor namespace. You implement a callback function that the mouse effectively calls whenever something interesting happens. It looks like this:

/// <summary>
/// Function receiving callback from mouse.
/// </summary>
/// <param name="pTouchMouseStatus">Values indicating status of mouse.</param>
/// <param name="pabImage">Bytes forming image, 13 rows of 15 columns.</param>
/// <param name="dwImageSize">Size of image, assumed to always be 195 (13x15).</param>
static void TouchMouseCallbackFunction(ref TOUCHMOUSESTATUS pTouchMouseStatus,
byte[] pabImage,
int dwImageSize)
{
...
}

As you can see, the image size is 13x15, always. You get the raw bytes that represent the image, 195 bytes long.

A console app outputing the center of mass as an X,Y coordinate

The first thing I was surprised to see was just HOW MUCH of the surface area of the mouse is covered and how high the resolution is. I was thinking, "13x15? That's so small, how could it tell me anything useful with such a limited structure?" Well, that's how clueless I was.

Of course, each of these bytes is giving a number between 0 and 255 so a lot of information can be extrapolated between two pixels with different values and you can get a pretty detailed picture of what's going on. How much? Well, before I move on to a graphical example, why not abuse ASCII as much as we can?

First, I'll output just the hex values with this code, then show what it looks like when I move my figures around the surface of the mouse.

Console.SetWindowSize(50,16);
Console.SetCursorPosition(0, 0);
StringBuilder sb = new StringBuilder(300);

for (Int32 y = 0; y < pTouchMouseStatus.m_dwImageHeight; y++)
{
// Iterate over columns.
for (Int32 x = 0; x < pTouchMouseStatus.m_dwImageWidth; x++)
{
// Get the pixel value at current position.
int pixel = pabImage[pTouchMouseStatus.m_dwImageWidth * y + x];
if (pixel > 0)
sb.AppendFormat("#{0:X2}", pixel);
else
sb.Append(" ");
}
sb.Append("\n");

}
Console.Write(sb.ToString());

Here's the result, animated.

Cool. But what about with color? I could use the default Console stuff and set the colors, or I could get direct access to the Console, use the Win32 APIs and what not, but I could also use Tim Sneath's ConsoleEx class that's on NuGet thanks to Anthony Mastrean! First, I'll "install-package ConsoleEx" to bring it in.

How about a mouse touch heat-map with color? I thought about building a whole "find closest color" map and trying to map 255 different colors, but then I just did this.

if (pixel > 230)
Console.BackgroundColor = ConsoleColor.Red;
else if (pixel > 128)
Console.BackgroundColor = ConsoleColor.Yellow;
else if (pixel > 90)
Console.BackgroundColor = ConsoleColor.Green;
else if (pixel > 45)
Console.BackgroundColor = ConsoleColor.Blue;
else if(pixel > 15)
Console.BackgroundColor = ConsoleColor.DarkBlue;
else
Console.BackgroundColor = ConsoleColor.Black;

ConsoleEx.WriteAt(x*2, y, " ");

And the result is this:

You can do the same thing in WPF, of course, with better bitmap support, but seriously, with a Console-based heatmap, who needs graphics? ;)

OK, fine, here's the sample code that takes the results of the TouchMouseSensorEventArgs and creates a grayscale image.

void TouchMouseSensorHandler(object sender, TouchMouseSensorEventArgs e)
{
// We're in a thread belonging to the mouse, not the user interface
// thread. Need to dispatch to the user interface thread.
Dispatcher.Invoke((Action<TouchMouseSensorEventArgs>)SetSource, e);
}

void SetSource(TouchMouseSensorEventArgs e)
{
// Convert bitmap from memory to graphic form.
BitmapSource source =
BitmapSource.Create(e.Status.m_dwImageWidth, e.Status.m_dwImageHeight,
105, 96,
PixelFormats.Gray8, null, e.Image, e.Status.m_dwImageWidth);

// Show bitmap in user interface.
SensorImage.Source = source;
}

And the result:

256 color grayscale heatmap of the Touch Mouse

One way to look at this is as if the surface of the mouse is a tiny Xbox Kinect. What kinds of gestures could I recognize and hook up to do stuff? Control WIndows, do custom stuff inside my application, browser, or whatever. Any ideas?

The downloads are here:

Enjoy!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

An analysis of SOPA and PIPA Protest "Blackout" HTML and CSS techniques

January 18, '12 Comments [28] Posted in Musings
Sponsored By

SOPA sucksMany popular sites are blacked out today in protest of two acts before Congress, known as the Protect IP Act (PIPA) in the Senate and the Stop Online Piracy Act (SOPA) in the House. Long story short, the legislators barely know how to email, much less understand what a DNS takedown is.

As I've been surfing around today bumping into SOAP sites, I noticed that every site is doing the blackout technique differently. This is interesting to me for a few reasons. First, things on the web aren't setup for an event like this. To shutdown your website for a day is one thing but to do it in such a way that your site presents a new temporary message without affecting search engines that have already indexed you is an interesting problem.

A number of people are changing their content for the day and some are rightfully concerned that this single day's protest will affect their search results. In fact, Google has announced via Google+ that they will be slowing the Googlebot in an attempt to mitigate the effects from the protest.

Put a giant DIV over your existing page

Wordpress.org had this clever technique. I hit the site, saw that it was black, then did a view source. However, when I did, the HTML source had the regular WordPress site that's always there. I used an element inspector as you can see below and started moving around. Note the blue outline that is outlining some phantom element.

WordPress › Blog Tool, Publishing Platform, and CMS - Windows Internet Explorer (106)

What's going on here? Let's blow it up using Tilt in Firefox.

image

Ah, see those three black buttons on the right? WordPress's original site is still under all that. They are floating a DIV over the top with a super large z-index.

If you change the CSS dynamically and change the z-indexes, (or just delete the DIVs) the actual site peeks out. Clever. A nice dramatic effect with minimal effort and effectively no SEO downside.

WordPress SOPA floats a giant DIV

However, the WordPress technique does kick off the process with JavaScript. What happens if you have JavaScript disabled? I assumed it just wouldn't work.

WordPress No JS

Nice! They've included a banner in the upper right corner using Pure CSS so there is still a visible protest on the page. A good compromise and a nice recognition of the six neckbeards that still surf the web without JavaScript. ;)

In my desperate search to learn everything I can about Justin Bieber, I turn to Wikipedia. I hit the site, I see Justin and then a curtain falls. Wikipedia's blackout page shows up AFTER a teasing flash of content.

Justin Bieber - Wikipedia

They've added a div at the VERY end of their page:

<div id="mw-sopaOverlay">
<div id="mw-sopaColumn">
<div id="mw-sopaHeadline">Imagine a World<br>Without Free Knowledge</div>
<div id="mw-sopaText"><p>For over a decade, we have spent millions of hours building the largest encyclopedia in human history. Right now, the U.S. Congress is considering legislation that could fatally damage the free and open Internet. For 24 hours, to raise awareness, we are blacking out Wikipedia. <a href="http://en.wikipedia.org/wiki/Wikipedia:SOPA_initiative/Learn_more" target="_blank">Learn more.</a></p></div><div id="mw-sopaAction">
<p class="mw-sopaActionHead">Contact your representatives.</p>
<div class="mw-sopaActionDiv">
<form action="/wiki/Special:CongressLookup">
<label for="zip">Your ZIP code:</label>
<input name="zip" size="5" type="text">
<input id="sopa-zipform-submit" value="Look up" type="submit">
</form>
</div>
</div>
</div>
</div>

At the very TOP of their page they have a bunch of positioning CSS. The flash happens because the DIV doesn't show up and get styled until the VERY end of the page gets loaded. Nice simple way to do it as SEO is so important to Wikipedia.

Actually Go Offline with an HTTP 503

The most impressive of the options, and arguably the most semantically correct, is the HTTP 503 Error. When you ask your browser for a page, it usually gets a 200 which means "OK, cool" or a 302 that means "temporarily over here" or even a 301 that means "permanently moved. Jimmy no live here! You no call back!"

An HTTP 503 means "Service Unavailable." I'd look it up on Wikipedia, but well, you know.

Notice in the Network tab picture below that they literally return a 503 result from the main HTTP GET. One could imagine them returning a 200 and just saying in the HTML it was a 503, but in this case they really used the web correctly. The Googlebot will visit the page, see the 503, assume the site is down temporarily and try again tomorrow.

Frankly, this is the technique everyone should have done. All the CSS and DIV work is clever, but not as easy , not as dramatic and certainly not as "correct."

503 Service Unavailable

Kudos to BoingBoing - a group that clearly gets the web - to use HTTP as it was designed to, to make their statement.

Redact by changing CSS Background Colors

From a design and stylistic perspective, Wired really nailed it. I love what they did here with the text intact and background colors in black.

Wired redacts text

You can hover over the text and see what's under it. No SEO affect, and still makes the point in a very dramatic way.

Zoom in on Wired Text Detail

Here's some of the their markup and CSS.

Wired.com censored class

They have a "censor" class" and apply it to a SPAN from a sopa.css.

Change an Image

My friends at DuckDuckGo changed their main image and linked to SOPA information. A simple change that won't affect SEO and is easily undone in the future.

image

 

Google did the same thing in fact, with simple markup.

image

Just:

<a href="//www.google.com/landing/takeaction/">
<img alt="Tell Congress: Please don't censor the web!"
title="Tell Congress: Please don't censor the web!"
border="0" height="196" src="/logos/2012/sopa12_hp.png"
width="445" style="padding-top:50px" id="hplogo">
</a>

Make you Wait or Click

Craigslist had an interstitial that blacked out their site with a link at the button to click to continue.

Craigslist against SOPA

Interestingly, Craigslist did their protest in the lowest-tech way possible. They literally just changed their first page. I suspect there will be a few days of SEO issues and we may see this page in search results.

Dramatic Animation

Today Firefox's about:home included a nice CSS animation that takes the background of the page from white to black to a nice effect.

Firefox blacked out in an animation

What other cool SOPA protest techniques have you seen today? You can add a SOPA widget to your own site with Grassroutes.


Sponsor: My thanks to DevExpress for sponsoring this week's feed: Introducing DXv2 – Welcome to the next generation of developer tools from DevExpress! Explore what’s new in the latest release.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Setting a Custom Icon for your External Drives in Windows Explorer

January 10, '12 Comments [25] Posted in Musings
Sponsored By

Ok, I like my icons. You arrange your desk perhaps, shuffle papers, I update things with totally unneeded icons. A while back I made, ahem, these awesome Visual Studio Command Prompt and PowerShell icons with Overlays because, frankly, pretty icons make life better.

I did these back in the day.

image

I even did a little one for the system menu, 'cause that's how I roll.

image

I just bought a nice 3 Terabyte Seagate USB3 External Drive that I'm very happy with, but I noticed that it had an awesome icon. This epic icon was was making my other drives jealous. This, of course, cannot stand, Dear Reader.

Ugly icons! Make it stop!

Easily fixed, friends.

Create a text file in the root of each drive called Autorun.inf with contents like this:

[autorun]
icon=\CustomIcon.ico

Any icon you like will do. You may need to reboot to see the changes. Now my externals are making my internals look bad!

Lovely Icons

Enjoy!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Easy steps to a mobile-friendly responsive design with an embedded YouTube video and a fluid resize

January 6, '12 Comments [11] Posted in HTML5 | Mobile | Open Source | Speaking
Sponsored By

imageI recently did a video with Rob Conery on how to be a better technical speaker and blogged about it. I wanted to put up a site for this video to give people more details and to make it easier for me to get the word about about the video separate from Tekpub. I went and bought http://speakinghacks.com and fired up WebMatrix to do a quick one-pager.

The idea was to spend only an hour on this from the moment I got the domain to a "complete" site. My requirements were:

  • An easy to remember domain name.
  • A site that looks kind of like my existing site, to keep the branding cohesive.
    • Cool I've got some existing CSS.
  • Site should look good on mobiles and tablets so I want to use responsive design.
    • Um...
  • A site that includes the free trailer for my video that plays on any device, including phones and tablets.
    • Um...
  • A site that resizes cleanly, rotates cleanly and the video thumbnail is always as large as possible.
    • Um...

First step, a decent starter template that works on Mobile

For stuff like this, I now always started with a decent boilerplate. There's even group knowledge and experience that there's little reason for me to New Blank HTML File these days.

There are a number of options out there, including two of my favorites:

There's others, and the nice thing is that you can get started with a template like this in minutes, not hours, so go find one that makes you happy.

I picked Skeleton, then with a little magic help from Jzy and a late night Skype (go visit his http://programmerryangosling.tumblr.com site, by the way!) we brought in a few elements from my currently blog template so the general look and feel is still there.

Add a YouTube video

There's a number of ways to embed a video that will use Silverlight or Flash if HTML5 video isn't available, including

  • VideoJS - A really nice and image-free HTML5 video player with a series of templates that can make it look like Vimeo or YouTube or others. Works everywhere.
  • SublimeVideo - Free and pretty but they have a few sign up things if you serve hundreds of thousands of views.
  • jMediaelement or "jme" - Clean, basic, simple, open. Flash fallback and semantic code.

These are all great but if you've already got your video up on YouTube, Vimeo, Blip or another video sharing site, you might just want to make that existing video embed resizable.

For that, you can use FitVid, a small jQuery plugin that will take effectively any video element and make it resizable and usable in a responsive design.

Originally Rob was using a Flash player called Flowplayer but I recommended that he save money by serving his video trailers on YouTube. That meant that I could serve my trailer video from YouTube. While I could then use the FitVid JavaScript to make the video resizable, it seemed like overkill to use some JavaScript to resize something that CSS should be handling for me.

Enter A List Apart and their article on Intrinsic Ratios for Video. The hardish part about resizing a video is maintaining the ratio, like 16:9 for example. As they say:

padding-bottom: 56.25%
To create a 16:9 ratio, we must divide 9 by 16 (0.5625 or 56.25%).

Anders M. Andersen has a nice clean example of this with the CSS, so I ended up with this markup. Only the embed-container class matters in this context. The other containers are used by Skeleton.

<div class="container">
<div class="sixteen columns">
<div class="embed-container">
<iframe id='player' src="http://www.youtube.com/embed/4U4TzAtyYs0" frameborder="0" allowfullscreen></iframe>
</div>
</div>
</div>

And this CSS:

.embed-container {    
position: relative;
padding-bottom: 56.25%; /* 16/9 ratio */
padding-top: 30px; /* IE6 workaround*/
height: 0;
overflow: hidden;
}

.embed-container iframe,
.embed-container object,
.embed-container embed {
position: absolute;
top: 0;
left: 0;
width: 100%;
height: 100%;
}

photoSo now, when you visit http://speakinghacks.com in your desktop browser or your mobile browser, you'll have a lovely experience.

Conclusion

I think I only spent about 2 hours total on this site. Taking advantage of existing CSS wisdom and standing on the shoulders of giants is absolutely the way to go for layouts. I'm totally sold on Responsive Design and am planning on including it on all my sites whenever possible.

Related Links

P.S. As an aside, if you bought the video, drop me a line if you liked it and maybe I can put your review on the site! Also feel free to write a review at SpeakerMix if you've seen me speak before.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Your Blog is The Engine of Community

January 5, '12 Comments [66] Posted in Blogging | Musings
Sponsored By

Photo by Steven Warburnton - Creative CommonsIn a time where there is much gnashing of teeth around the meaning of community, what being on the "inside" vs. the "outside" means, I want to take a moment to remind my fellow blog writers, blog readers, blog commenters what makes it all work.

You.

Not a secret society or old boy's network, not a select few or someone knighted by The Queen. It's the nameless, faceless web search result that makes community work.

I search all the time for help on the internet. I find blogs, tweets, Stack Overflow, MSDN and more. More often than not when I find the answer I seek it's on YOUR blog, not mine. Often it's not on a big company employee's blog or that of the chosen few. The answer was put out on a blog, without ask of payment or recognition, by a 25-year old Persian student, or a 60-year old exploring .NET, or a high school student with a passion for open source.

I, and this blog, was that random search result for at least 5 of the last 10 years. Someone searches for help and finds my little corner of the internet. Write a few blog posts a week, with useful content, consistently, for ten years. Then write some more. All free, all because you feel good putting it out there.

I would encourage you all to blog more. Tweet less. Blogs are owned by you. They are easily found, easily linked to, and great conversations happen with great blog posts. The river of social media rushes on and those conversations are long forgotten. A great blog post is forever. Today's real-time social media is quickly forgotten.

Don't be a meme, but a movement.

Blog your opinions. Blog your cool project, or your latest useful function or library. Don't blog if it feels like work. Blog and get excited when someone comments. Often the comments are more fun and more useful than the post itself. Be passionate, but not rude. Point out failings, but suggest solutions. Organize. Invent.

Be constructive, be helpful, be kind. Make your blog posts not too long, not too short, not too stream-of-consciousness and not too terse. Remember your elementary writing classes. Have a thesis, make your argument, restate your thesis.

Share because you want to. Share because you want to help, but also because you want to help yourself. Share not for the recognition but for the love of teaching.

It takes a village, dear reader, to be a community. It's you, and me and no one in between. Now, go write, create, commit.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.