Scott Hanselman

The Weekly Source Code 26 - LINQ to Regular Expressions and Processing in Javascript

May 10, '08 Comments [11] Posted in ASP.NET | Javascript | LINQ | Silverlight | Source Code
Sponsored By

I've been getting more and more interested in how folks extend their applications using plugins and things. In my new ongoing quest to read source code to be a better developer, Dear Reader, I present to you twenty-sixth (half a year!) in a infinite number of posts of "The Weekly Source Code."

Sometimes when I read code, I kick myself (mentally) and say "Man, I should have thought of that!" Then I realize I'm not nearly as good a programmer as I think I am, and then I just let the source just wash over my brain.

Here's some source by smart people I've been reading this week that I should have thought of. ;) Coincidentally they are both examples of languages ported or re-imagined in another language.

"Processing" in JavaScript

When I say "Processing" I mean the open-source Java-based visualization language from http://processing.org/. Jeff calls it "more akin to sketching than coding" while I say it's sketching with code! Jeff yearns: "for the day when web pages are regularly illustrated with the kind of beautiful, dynamic visualizations that Ben Fry creates."

molten2

Well, John Resig, arguably already considered one of the best JavaScript coders on the planet after he gave us the tour de force that is JQuery, has ported Processing to Javascript and gives us Processing.js.

You can interact with it in two ways. First, as an elegant and tight Javascript API:

var p = Processing(CanvasElement);
p.size(100, 100);
p.background(0);
p.fill(255);
p.ellipse(50, 50, 50, 50);

Or, you can tunnel the actual Processing language like this:

Processing(CanvasElement, "size(100, 100); background(0);" + "fill(255); ellipse(50, 50, 50, 50);");

clockThis release is specifically targeted to Firefox3, Opera 9.5 and the Webkit Nightlies (Safari) - all unreleased, beta browsers. I'm going to try it under the DLR with Javascript in Silverlight. Heh heh.

Here are his demos. Remember, these don't work in IE7.

There's a load of demos, but here's a powerful one. A working clock in 17 lines of code.

void setup() {
size(200, 200);
stroke(255);
smooth();
}
void draw() {
background(0);
fill(80);
noStroke();
// Angles for sin() and cos() start at 3 o'clock;
// subtract HALF_PI to make them start at the top
ellipse(100, 100, 160, 160);
float s = map(second(), 0, 60, 0, TWO_PI) - HALF_PI;
float m = map(minute(), 0, 60, 0, TWO_PI) - HALF_PI;
float h = map(hour() % 12, 0, 12, 0, TWO_PI) - HALF_PI;
stroke(255);
strokeWeight(1);
line(100, 100, cos(s) * 72 + 100, sin(s) * 72 + 100);
strokeWeight(2);
line(100, 100, cos(m) * 60 + 100, sin(m) * 60 + 100);
strokeWeight(4);
line(100, 100, cos(h) * 50 + 100, sin(h) * 50 + 100);
}

His code leans heavily on the Canvas which is why IE7 doesn't work. Much of the processing.js file is mapping from one API (the processing API) to Javascript constructs, usually canvas ones. For example, making a point(x,y) is:

  p.point = function point( x, y )
{
var oldFill = curContext.fillStyle;
curContext.fillStyle = curContext.strokeStyle;
curContext.fillRect( Math.round( x ), Math.round( y ), 1, 1 );
curContext.fillStyle = oldFill;
}

Note the rectangle that is 1 by 1. That's funny, but that's the life an API mapper. Remind me someday to tell you, Dear Reader, how I got filled pie charts working on an Original Palm Pilot that not only didn't support Put/GetPixel but didn't have floating point math. That was a hoot.

Anyway, one really good example of this guy's clean cleverness is the triangle function. Remember, this is a processing function and he's not only got to implement it, but also make the building blocks for doing it cleanly.

To start:

  p.triangle = function triangle( x1, y1, x2, y2, x3, y3 )
{
p.beginShape();
p.vertex( x1, y1 );
p.vertex( x2, y2 );
p.vertex( x3, y3 );
p.endShape();
}

Obvious, right? Well, not really, considering that the 2D Canvas doesn't have any of those three higher-level methods. Begin and EndShape are fairly clean. However, he had to implement a nice Fill, Stroke and ClosePath to do this cleanly.

  p.beginShape = function beginShape( type )
{
curShape = type;
curShapeCount = 0;
}

p.endShape = function endShape( close )
{
if ( curShapeCount != 0 )
{
curContext.lineTo( firstX, firstY );

if ( doFill )
curContext.fill();

if ( doStroke )
curContext.stroke();

curContext.closePath();
curShapeCount = 0;
pathOpen = false;
}

if ( pathOpen )
{
curContext.closePath();
}
}

It's about four layers deep, each primitive building on the next until he gets a nice clean triangle implementation, but then he can use it for quad() and it the same method handles bezierVertex as well. It would do you well to FireBug your way through his code. It's a wonderful fun way to re-learn Javascript from a gentleman who knows what he's doing.

LINQ to RegEx and Fluent Regular Expressions

I was trying to re-re-re-learn Regular Expressions again this week for a small task. It's funny how Regular Expressions are the first thing to leave my brain even though there are a bunch of Regular Expression Tools out there. Josh Flanagan came up with a Fluent Interface for Regular Expressions like:

Regex socialSecurityNumberCheck = new Regex(@"^\d{3}-?\d{2}-?\d{4}$");

would look like this:

Regex socialSecurityNumberCheck = new Regex(Pattern.With.AtBeginning 
.Digit.Repeat.Exactly(3)
.Literal("-").Repeat.Optional
.Digit.Repeat.Exactly(2)
.Literal("-").Repeat.Optional
.Digit.Repeat.Exactly(4)
.AtEnd);

It took me a second to like this. OK, it took me a while. Breathe for a minute, and read it out loud. It kind of makes sense, actually, although there is a reasonable argument against in the comments of Josh's post:

[It] strikes me that a user of this library needs to learn a fairly complex syntax which is almost as far from "plain english" as regex, when they could simply learn how to do regex.

Sure, but it's fun to try new things. If you look a his source, it's really just a really smart string concatenator. I think it would actually be a very interesting way to teach or learn regular expressions, especially if you're a casual RegEx'er like me.

Krzysztof Ko┼║mic created a similar API in 2007. His fluent interface over RegEx looks like this:

Pattern pattern = Pattern.Define().
As("Kot".Count(Times.AtLeast(2))).
FollowedBy(Any.Except('a','b','c')).
Start(At.BeginingOfStringOrLine);

Then Roy Osherove took Josh's API further and took Josh's Fluent Interface to RegEx from 2006 and applied a LINQ query syntax , creating in the process, LINQ to Regex.

Here's Roy's example:

public void FindEmailUsingPattern()
{
var query = from match in
RegexQuery.Against("sdlfjsfl43r3490r98*(*Email@somewhere.com_dakj3j")
where match.Word.Repeat.AtLeast(1)
.Literal("@")
.Word.Repeat.AtLeast(1)
.Literal(".")
.Choice.Either(
Pattern.With.Literal("com"),
Pattern.With.Literal("net"))
.IsTrue()
select match;
foreach (var match in query)
{
Assert.AreEqual("Email@somewhere.com",match.Value);
}
}

After the "from match in", the simple heart of it is Roy's static Against() call that returns a RegexQuery that is IEnumerable of Match, thereby supporting the foreach later on:

namespace Osherove.LinqToRegex 
{
public class RegexQuery : IEnumerable
{
private readonly string input;
private object lastPatternRetVal;
private RegexQuery(string input)
{
this.input = input;
}
public static RegexQuery Against(string input)
{
return new RegexQuery(input);
}

private string _regex;
public RegexQuery Where(Expression<func><pattern,bool> predicate)
{
_regex = new PatternVisitor().VisitExpression(predicate).ToString();
return this;
}

public RegexQuery Select<t>(Expression<func><pattern,t> selector)
{
return this;
}
#region IEnumerable Members

IEnumerator IEnumerable.GetEnumerator()
{
return ((IEnumerable)this).GetEnumerator();
}

public IEnumerator GetEnumerator()
{
MatchCollection matches = Regex.Matches(input, _regex);
foreach (Match found in matches)
{
yield return found;
}
}
}
}

You can find all the source for Roy's project up at his assembla.com project site and Josh's source is on his blog. It is worth noting, though that you can combine LINQ Queries with Regular Expressions without any tricks because Matches are returned in a MatchCollection an LINQ loves things that are IEnumerable.

You can use LINQ projections to pull objects out of a collection of matches like:

    List<yourType> = (from Match m in matches
select new YourType
{
Id = m.Groups[1].Value,
Something = m.Groups[2].Value
}).ToList();

So, we've got two sides of the coin here. First, the creation of the Regular Expression. That can be the standard way, or with a fluent interface. Either way, you end up with a string. Second, you've got the extraction of the information. Most often you'll care about the MatchCollection that comes back. You'll usually want to pull information out, so while you're foreach'ing your way over the collection, you can use LINQ to create an object projection that's chopped up and sorted and grouped all with one query, regardless of how you created the query in the first place.

Choice is good.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

TypeConverters: There's not enough TypeDescripter.GetConverter in the world

May 10, '08 Comments [12] Posted in Source Code
Sponsored By

While reading source today, I saw some code in the wild today that looked roughly like this (not the actual code):

type = typeof(T);
if (type == typeof(Boolean))
{
returnValue = (T)((object)Convert.ToBoolean(value));
}
else if (type == typeof(String))
{
returnValue = (T)((object)value);
}
else if (type == typeof(Int16))
{
returnValue = (T)((object)Convert.ToInt16(value));
}
else if (type == typeof(Int32))
{
returnValue = (T)((object)Convert.ToInt32(value));
}
//...and on and on for a dozen+ types

You get the idea. This isn't that uncommon, I've seen it more of than not. The person who is writing it usually knows it's bad 50% of the time, but it's always a trade-off between figuring out the right search query (a tough one, in this case!) and knowing what to look for in MSDN.

This brings up one of my favorite classes in the BCL, the TypeDescriptor class. A method like this:

public static T GetTfromString<T>(string mystring)
{
var foo = TypeDescriptor.GetConverter(typeof(T));
return (T)(foo.ConvertFromInvariantString(mystring));
}

...would allow all that switch/if/else to go away replaced by:

bool b = GetTfromString("true");

You'd probably want to expand the method with checks to see if T was in fact, System.Type, or System.String, but you get the idea.

image There are lots of standard converters like EnumConvertor, ColorConvertor, and more, all waiting for you in System.ComponentModel as seen in the image at right.

Not only that, but you can make your own TypeConverters and spread the love. They are used extensively when displaying types in Property Grids, in WinForms, or in XAML. You can also use them in PowerShell as a better way to present your objects as strings when "casting" in PowerShell.

Jesse Liberty has a fine example showing how to use Type Convertors in Silverlight to enable you to put complex types in XAML attributes.

From MSDN:

To implement a simple type converter that can translate a string to a [Type]

  1. Define a class that derives from TypeConverter.

  2. Override the CanConvertFrom method that specifies which type the converter can convert from. This method is overloaded.

  3. Override the ConvertFrom method that implements the conversion. This method is overloaded.

  4. Override the CanConvertTo method that specifies which type the converter can convert to. It is not necessary to override this method for conversion to a string type. This method is overloaded.

  5. Override the ConvertTo method that implements the conversion. This method is overloaded.

  6. Override the IsValid method that performs validation. This method is overloaded.

That's it, and when that Type was decorated with an attribute like:

[TypeConverter(typeof(MyNewTypeConverter))]

Then that new TypeConverter would be picked up and used by the line of code I showed before. Much better than a switch or pile of if/elses, I think.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Introducing RockScroll

May 9, '08 Comments [64] Posted in Musings | Tools
Sponsored By

When I came to Microsoft I saw a really cool tool being used internally and immediately wanted to work with the author, Rocky Downs (who is blogless, but exceeding talented), to get it released. Fast forward to now, plus an installer, and here it is.

The basic (as in "only") idea is that RockScroll extends the scrollbar in Visual Studio to show a syntax highlighted thumbnail view of your source. This is really useful for those excessively long source code files you know you have. It's just one DLL and you can turn it off from Tools|AddIns just by un-checking the checkbox.

Enjoy!

RockScroll

works-on-my-machine-starburst Works On My Machine Disclaimer: This is released with exactly zero warranty or support. If it deletes files or kills your family pet, you have been warned. It might work great, and it might not. It hasn't been tested against the myriad of other VS Add-Ins, but it works on my machine in both VS2005 and VS2008. It does look a little odd next to Resharper, a tool that also adds a scrollbar. Good luck.

A reader, Hans, says: "I have found that (not suprisingly) RockScroll works very well with split windows, if you bind a shortcut key to the "Window.Split" command." for those of you having trouble with split windows.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Hanselminutes Podcast 110 - Microsoft Research: Spec#

May 8, '08 Comments [3] Posted in Podcast | Programming
Sponsored By

My one-hundred-and-tenth podcast is up. This episode was recorded at the ALT.NET Open Spaces Conference in Seattle a few weeks back. I got to sit down with two gentlemen from Microsoft Research, Mike and Rustan, and talk about Spec# after their presentation at the conference.

What's Spec#? Currently it's a new language that is essentially extensions to C# 2.0. For example:

class ArrayList { public virtual void Insert(int index , object value)
requires 0 <= index && index <= Count;
requires !IsReadOnly && !IsFixedSize;
{ . . . }

See all that new stuff BEFORE the first curly brace? That's where you specify preconditions that you want enforced. You can also specify postconditions:

ensures Count == old(Count) + 1;
ensures value == this[index ];
ensures Forall{int i in 0 : index ; old(this[i]) == this[i]};
ensures Forall{int i in index : old(Count); old(this[i]) == this[i + 1]};

There's LOTS more, including a pretty extraordinary static analysis that runs in the background, kind of like the grammar checker in Word. Word has the spelling checker that turns errors red with squiggly underlines, and the grammar checker that turns grammatical errors green with squiggly underlines. Spec# will turn static errors into squiggles while you type.

They've even added specifications for the Base Class Library (BCL) that have previously only existed in documentation.

UseMscorlibContracts - Microsoft Visual Studio

BTW, if you want to see horribly shaky cellphone video of their presentation, you can see it at Greg Young's blog. He's posted recently about Spec# as well. If you don't like videos or podcasts, you can also read the Spec# Research Paper (PDF).

How can you get involved? We need to convince that folks on the C# team that Spec# is a valuable thing and that we'd like it sooner than later. Go over to Charlie Calvert's blog and email him that you want Spec# and why!

Subscribe: Subscribe to Hanselminutes Subscribe to my Podcast in iTunes

If you have trouble downloading, or your download is slow, do try the torrent with µtorrent or another BitTorrent Downloader.

Do also remember the complete archives are always up and they have PDF Transcripts, a little known feature that show up a few weeks after each show.

Telerik is our sponsor for this show.

Check out their UI Suite of controls for ASP.NET. It's very hardcore stuff. One of the things I appreciate about Telerik is their commitment to completeness. For example, they have a page about their Right-to-Left support while some vendors have zero support, or don't bother testing. They also are committed to XHTML compliance and publish their roadmap. It's nice when your controls vendor is very transparent.

As I've said before this show comes to you with the audio expertise and stewardship of Carl Franklin. The name comes from Travis Illig, but the goal of the show is simple. Avoid wasting the listener's time. (and make the commute less boring)

Enjoy. Who knows what'll happen in the next show?

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

RFC: OpenTweets - Why is Microblogging centralized?

May 3, '08 Comments [21] Posted in Musings
Sponsored By

I'm thoroughly enjoying Twitter (follow me!) Apparently others are enjoying it also as there's a funny estimate that it's costing $14 billion in lost productivity (via a back of envelope calculation). There is about a million people on Twitter and maybe 60k adding each a month. Certainly it'll be blocked by most corporate firewalls soon for just this reason.

Twitter is also down ALL the time and it's been having weekly (daily?) scale problems for a YEAR, culminating in rumors that the development team is leaving Ruby on Rails that have been denied by Evan Williams (via a "Tweet," of course). However, that doesn't change the fact that Twitter is down so often there is a site dedicated to "Twitter Down Art" showcasing all the pictures that Twitter puts up when their sites are down. Google changes their art monthly and during holidays, but never goes down. Twitter goes down so often that they use that as an opportunity to change their art! So far, in this era of transparency, no one has explained in technical (or any) terms what the problem with Twitter is and folks are getting impatient. I think that it would be a great PR and Karmic move to just start a Twitter Technical Blog and share the crazy IT problem of the day. Surely someone is sleeping on a cot next to the TweetBoxes and has a story to tell.

There's another rumor that Twitter is worth about $150M. Seems to me that the second bubble hasn't popped. I'm certainly no business man, but we can build this better ourselves.

Twitter is centralized, which is insane (unless you're Google or Amazon or Live.com, etc, which are actually distributed in their own special way).

Twitter is IRC, it's IM, it's chatrooms, but ultimately it's just microblogging with a multi-format API. Here's the XML for my Twitter Account but it looks nicer as Atom.

But I already have a blog! Why use a service when I already host a blog? I hooked up Microsummaries to DasBlog two years ago. There's clearly precedent.

Two folks I know already have Twitter Backup Feeds. Both LazyCoder (backup twitstream) and Dave Winer (twitstream) have Backup Twitter Feeds. Dave takes it even farther and makes his twistream discoverable via a link in his blog:

<link rel="twitstream" href="http://twitter.scripting.com/daveRss.xml" />

With backups like these, who needs Twitter? There is already a very responsive group of Twitter client applications some of which, like twhirl cross-post to Pownce and Jaiku, two Twitter competitors. Russell Beattie has an idea for an Open Twitter Server called Peep using Jabber.

I propose that we distribute Twitter into generalized spec for microblogging and just use RSS or Jabber for the transport. I think the very-fast-rev'ing Twitter clients would support it. 

Thoughts? Should Twitter be a service or should Microblogging be an Open Thing?

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.