Scott Hanselman

The Weekly Source Code 26 - LINQ to Regular Expressions and Processing in Javascript

May 10, '08 Comments [11] Posted in ASP.NET | Javascript | LINQ | Silverlight | Source Code
Sponsored By

I've been getting more and more interested in how folks extend their applications using plugins and things. In my new ongoing quest to read source code to be a better developer, Dear Reader, I present to you twenty-sixth (half a year!) in a infinite number of posts of "The Weekly Source Code."

Sometimes when I read code, I kick myself (mentally) and say "Man, I should have thought of that!" Then I realize I'm not nearly as good a programmer as I think I am, and then I just let the source just wash over my brain.

Here's some source by smart people I've been reading this week that I should have thought of. ;) Coincidentally they are both examples of languages ported or re-imagined in another language.

"Processing" in JavaScript

When I say "Processing" I mean the open-source Java-based visualization language from http://processing.org/. Jeff calls it "more akin to sketching than coding" while I say it's sketching with code! Jeff yearns: "for the day when web pages are regularly illustrated with the kind of beautiful, dynamic visualizations that Ben Fry creates."

molten2

Well, John Resig, arguably already considered one of the best JavaScript coders on the planet after he gave us the tour de force that is JQuery, has ported Processing to Javascript and gives us Processing.js.

You can interact with it in two ways. First, as an elegant and tight Javascript API:

var p = Processing(CanvasElement);
p.size(100, 100);
p.background(0);
p.fill(255);
p.ellipse(50, 50, 50, 50);

Or, you can tunnel the actual Processing language like this:

Processing(CanvasElement, "size(100, 100); background(0);" + "fill(255); ellipse(50, 50, 50, 50);");

clockThis release is specifically targeted to Firefox3, Opera 9.5 and the Webkit Nightlies (Safari) - all unreleased, beta browsers. I'm going to try it under the DLR with Javascript in Silverlight. Heh heh.

Here are his demos. Remember, these don't work in IE7.

There's a load of demos, but here's a powerful one. A working clock in 17 lines of code.

void setup() {
size(200, 200);
stroke(255);
smooth();
}
void draw() {
background(0);
fill(80);
noStroke();
// Angles for sin() and cos() start at 3 o'clock;
// subtract HALF_PI to make them start at the top
ellipse(100, 100, 160, 160);
float s = map(second(), 0, 60, 0, TWO_PI) - HALF_PI;
float m = map(minute(), 0, 60, 0, TWO_PI) - HALF_PI;
float h = map(hour() % 12, 0, 12, 0, TWO_PI) - HALF_PI;
stroke(255);
strokeWeight(1);
line(100, 100, cos(s) * 72 + 100, sin(s) * 72 + 100);
strokeWeight(2);
line(100, 100, cos(m) * 60 + 100, sin(m) * 60 + 100);
strokeWeight(4);
line(100, 100, cos(h) * 50 + 100, sin(h) * 50 + 100);
}

His code leans heavily on the Canvas which is why IE7 doesn't work. Much of the processing.js file is mapping from one API (the processing API) to Javascript constructs, usually canvas ones. For example, making a point(x,y) is:

  p.point = function point( x, y )
{
var oldFill = curContext.fillStyle;
curContext.fillStyle = curContext.strokeStyle;
curContext.fillRect( Math.round( x ), Math.round( y ), 1, 1 );
curContext.fillStyle = oldFill;
}

Note the rectangle that is 1 by 1. That's funny, but that's the life an API mapper. Remind me someday to tell you, Dear Reader, how I got filled pie charts working on an Original Palm Pilot that not only didn't support Put/GetPixel but didn't have floating point math. That was a hoot.

Anyway, one really good example of this guy's clean cleverness is the triangle function. Remember, this is a processing function and he's not only got to implement it, but also make the building blocks for doing it cleanly.

To start:

  p.triangle = function triangle( x1, y1, x2, y2, x3, y3 )
{
p.beginShape();
p.vertex( x1, y1 );
p.vertex( x2, y2 );
p.vertex( x3, y3 );
p.endShape();
}

Obvious, right? Well, not really, considering that the 2D Canvas doesn't have any of those three higher-level methods. Begin and EndShape are fairly clean. However, he had to implement a nice Fill, Stroke and ClosePath to do this cleanly.

  p.beginShape = function beginShape( type )
{
curShape = type;
curShapeCount = 0;
}

p.endShape = function endShape( close )
{
if ( curShapeCount != 0 )
{
curContext.lineTo( firstX, firstY );

if ( doFill )
curContext.fill();

if ( doStroke )
curContext.stroke();

curContext.closePath();
curShapeCount = 0;
pathOpen = false;
}

if ( pathOpen )
{
curContext.closePath();
}
}

It's about four layers deep, each primitive building on the next until he gets a nice clean triangle implementation, but then he can use it for quad() and it the same method handles bezierVertex as well. It would do you well to FireBug your way through his code. It's a wonderful fun way to re-learn Javascript from a gentleman who knows what he's doing.

LINQ to RegEx and Fluent Regular Expressions

I was trying to re-re-re-learn Regular Expressions again this week for a small task. It's funny how Regular Expressions are the first thing to leave my brain even though there are a bunch of Regular Expression Tools out there. Josh Flanagan came up with a Fluent Interface for Regular Expressions like:

Regex socialSecurityNumberCheck = new Regex(@"^\d{3}-?\d{2}-?\d{4}$");

would look like this:

Regex socialSecurityNumberCheck = new Regex(Pattern.With.AtBeginning 
.Digit.Repeat.Exactly(3)
.Literal("-").Repeat.Optional
.Digit.Repeat.Exactly(2)
.Literal("-").Repeat.Optional
.Digit.Repeat.Exactly(4)
.AtEnd);

It took me a second to like this. OK, it took me a while. Breathe for a minute, and read it out loud. It kind of makes sense, actually, although there is a reasonable argument against in the comments of Josh's post:

[It] strikes me that a user of this library needs to learn a fairly complex syntax which is almost as far from "plain english" as regex, when they could simply learn how to do regex.

Sure, but it's fun to try new things. If you look a his source, it's really just a really smart string concatenator. I think it would actually be a very interesting way to teach or learn regular expressions, especially if you're a casual RegEx'er like me.

Krzysztof Koźmic created a similar API in 2007. His fluent interface over RegEx looks like this:

Pattern pattern = Pattern.Define().
As("Kot".Count(Times.AtLeast(2))).
FollowedBy(Any.Except('a','b','c')).
Start(At.BeginingOfStringOrLine);

Then Roy Osherove took Josh's API further and took Josh's Fluent Interface to RegEx from 2006 and applied a LINQ query syntax , creating in the process, LINQ to Regex.

Here's Roy's example:

public void FindEmailUsingPattern()
{
var query = from match in
RegexQuery.Against("sdlfjsfl43r3490r98*(*Email@somewhere.com_dakj3j")
where match.Word.Repeat.AtLeast(1)
.Literal("@")
.Word.Repeat.AtLeast(1)
.Literal(".")
.Choice.Either(
Pattern.With.Literal("com"),
Pattern.With.Literal("net"))
.IsTrue()
select match;
foreach (var match in query)
{
Assert.AreEqual("Email@somewhere.com",match.Value);
}
}

After the "from match in", the simple heart of it is Roy's static Against() call that returns a RegexQuery that is IEnumerable of Match, thereby supporting the foreach later on:

namespace Osherove.LinqToRegex 
{
public class RegexQuery : IEnumerable
{
private readonly string input;
private object lastPatternRetVal;
private RegexQuery(string input)
{
this.input = input;
}
public static RegexQuery Against(string input)
{
return new RegexQuery(input);
}

private string _regex;
public RegexQuery Where(Expression<func><pattern,bool> predicate)
{
_regex = new PatternVisitor().VisitExpression(predicate).ToString();
return this;
}

public RegexQuery Select<t>(Expression<func><pattern,t> selector)
{
return this;
}
#region IEnumerable Members

IEnumerator IEnumerable.GetEnumerator()
{
return ((IEnumerable)this).GetEnumerator();
}

public IEnumerator GetEnumerator()
{
MatchCollection matches = Regex.Matches(input, _regex);
foreach (Match found in matches)
{
yield return found;
}
}
}
}

You can find all the source for Roy's project up at his assembla.com project site and Josh's source is on his blog. It is worth noting, though that you can combine LINQ Queries with Regular Expressions without any tricks because Matches are returned in a MatchCollection an LINQ loves things that are IEnumerable.

You can use LINQ projections to pull objects out of a collection of matches like:

    List<yourType> = (from Match m in matches
select new YourType
{
Id = m.Groups[1].Value,
Something = m.Groups[2].Value
}).ToList();

So, we've got two sides of the coin here. First, the creation of the Regular Expression. That can be the standard way, or with a fluent interface. Either way, you end up with a string. Second, you've got the extraction of the information. Most often you'll care about the MatchCollection that comes back. You'll usually want to pull information out, so while you're foreach'ing your way over the collection, you can use LINQ to create an object projection that's chopped up and sorted and grouped all with one query, regardless of how you created the query in the first place.

Choice is good.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

How to set an IIS Application or AppPool to use ASP.NET 3.5 rather than 2.0

April 1, '08 Comments [42] Posted in ASP.NET | IIS | LINQ
Sponsored By

A question that comes up a lot is this: How do I tell my IIS Application or Virtual Directory to use ASP.NET 3.5 rather than ASP.NET 2.0?

Folks often go into the IIS6 or IIS7 manager and setup an AppPool and see a properties dialog like this, and when the pull down the option, they often expect to see .NET 3.0 and .NET 3.5 in the list and it's not there, and they freak out, rightfully so.

image

Here's an explanation of why this is confusing, and hopefully by the end, it won't be confusing anymore.

Marketing

This is where marketing and reality part ways. I didn't initially like the naming, because I assumed that each major version of the Framework meant a new CLR, but it's growing on me as I understand it more. I get now why they named them this way. Additional fundamentally new functionality has to be named something.

image

.NET 2.0

The meat of .NET is in %windir%\Microsoft.NET\Framework\v2.0.50727. That's where, along with the GAC (Global Assembly Cache) all the libraries and compilers you know and love live. When folks ask "where is .NET" I usually start here.

image

.NET 3.0

The addition of .NET 3.0 didn't mean new compilers or a new CLR. Instead, it's three major new libraries: WCF (Windows Communication Foundation née Indigo), WPF (Windows Presentation Foundation née Avalon) and Windows Workflow or WF.

image

Bottom line: Installing .NET 3.0 doesn't fundamentally change your system in any way you should fear. Your 2.0 apps still run on a system with 3.0 installed. They are 2.0 apps using the 2.0 compilers and 2.0 CLR.

imageTry this. If you go into Visual Studio and File | New | Project (or Web Site), take note of the drop down in the upper right corner.

Select ".NET Framework 3.0" and make a new "WCF Service" take a look at the web.config. Note that it's pretty conventional, and should look like a typical .NET 2.0 ASP.NET application with a few additional.

Basically, remember Framework version != CLR Version. If you configured an IIS Application to use .NET 2.0, you're talking about the 2.0 CLR. WCF Applications use the .NET 2.0 CLR with the new 3.0 WCF libraries.

  • .NET Framework 1.x = CLR 1.x
  • .NET Framework 2.0 = CLR 2.0
  • .NET Framework 3.0 = CLR 2.0
  • .NET Framework 3.5 = CLR 2.0 + (C# 3.0 | VB9)

You can also use the new 3.5 compilers and the 3.0 libraries, of course as well. Each subsumes the previous as seen in Tim Sneath's fine stacked diagram above.

image

In your new app's web.config, there's a <system.serviceModel> section that is WCF specific, but otherwise the web.config looks pretty 2.0 typical.

.NET 3.5

The marketing term ".NET Framework 3.5" refers to a few things. First, LINQ, which is huge, and includes new language compilers for C# and VB. Second, the REST support added to Windows Communication Foundation, as well as, third, the fact that ASP.NET AJAX is included, rather than a separate download as it was before in ASP.NET 2.0.

There's a few other things in .NET 3.5, like SP1 of .NET 2.0 to fix bugs, but one way to get an idea of what's been added in .NET 3.5 is to look in c:\windows\assembly. Here's just the 3.5 versioned assemblies in the GAC (Global Assembly Cache).

image

Also, looking in %windir%\Microsoft.NET\Framework\v3.5 we can see the new compilers, MSBuild Target files, etc.

image 

So, getting to answering the original question, try this experiment.

Go into Visual Studio and make a .NET 2.0 Web Site. Once it's loaded up, note your web.config. Next, right-click on the project and select Properties. Under Build, select 3.5 Framework.

image

Now, load up your web.config and notice the changes that just occurred. There's some new handlers that are added to support Ajax and some new ASP.NET Features, but the really important stuff is the <system.codedom> and the newly added assemblies in the assemblies section.

    <compilation debug="false">
<assemblies>
<add assembly="System.Core, Version=3.5.0.0, Culture=neutral, PublicKeyToken=B77A5C561934E089"/>
<add assembly="System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"/>
<add assembly="System.Xml.Linq, Version=3.5.0.0, Culture=neutral, PublicKeyToken=B77A5C561934E089"/>
<add assembly="System.Data.DataSetExtensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=B77A5C561934E089"/>
</assemblies>
</compilation>
<system.codedom>
<compilers>
<compiler language="c#;cs;csharp" extension=".cs"
type="Microsoft.CSharp.CSharpCodeProvider,System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"
warningLevel="4">
<providerOption name="CompilerVersion" value="v3.5"/>
<providerOption name="WarnAsError" value="false"/>
</compiler>
<compiler language="vb;vbs;visualbasic;vbscript" extension=".vb"
type="Microsoft.VisualBasic.VBCodeProvider, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"
warningLevel="4">
<providerOption name="CompilerVersion" value="v3.5"/>
<providerOption name="OptionInfer" value="true"/>
<providerOption name="WarnAsError" value="false"/>
</compiler>
</compilers>
</system.codedom>

There's the magic. Well, not really magic, and there's nothing hidden. This where your web site is told what version of the compiler to use, and the new supporting libraries.

This is where you tell ASP.NET to use .NET 3.5, not in IIS. IIS AppPools know about CLR versions, not Framework and compiler versions, those are set by the application.

Now, this is just my opinion, but I like to name my AppPools in IIS like this...

image

...which looks the way my brain thinks it is even though it's not reality. I keep my 1.1, 2.0 and 3.5 applications all running under the same instance of IIS, but each in their own AppPool. Note again, the there's nothing keeping me from having 3.5 apps under my 2.0 AppPool, I just keep them tidy on my own.

Technorati Tags: ,,,

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Hanselminutes Podcast 105 - Rocky Lhotka on Data Access Mania, LINQ and CSLA.NET

March 28, '08 Comments [7] Posted in LINQ | Podcast | Programming
Sponsored By

rocky0005-120wMy one-hundred-and-fifth podcast is up. I got a chance to sit down with Rocky Lhotka (blog) and talk about the direction data access, business objects and multi-tier development are going, as well as where he things LINQ fits into his view of CSLA.NET. CSLA.NET is Rocky's application development framework that supports his multi-tiered view of business application development.

Subscribe: Subscribe to Hanselminutes Subscribe to my Podcast in iTunes

If you have trouble downloading, or your download is slow, do try the torrent with µtorrent or another BitTorrent Downloader.

Do also remember the complete archives are always up and they have PDF Transcripts, a little known feature that show up a few weeks after each show.

Telerik is our sponsor for this show.

Check out their UI Suite of controls for ASP.NET. It's very hardcore stuff. One of the things I appreciate about Telerik is their commitment to completeness. For example, they have a page about their Right-to-Left support while some vendors have zero support, or don't bother testing. They also are committed to XHTML compliance and publish their roadmap. It's nice when your controls vendor is very transparent.

As I've said before this show comes to you with the audio expertise and stewardship of Carl Franklin. The name comes from Travis Illig, but the goal of the show is simple. Avoid wasting the listener's time. (and make the commute less boring)

Enjoy. Who knows what'll happen in the next show?

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Getting LINQ to SQL and LINQ to Entities to use NOLOCK

March 18, '08 Comments [23] Posted in LINQ
Sponsored By

I was visiting a local company where a bunch of friends work and they asked how to get LINQ to SQL to use NOLOCK. They were explicitly asking how to get the SQL generated by LINQ to SQL to add the NOLOCK hints to the end.

However, with NOLOCK (even though "everyone" has used it at some point or another) is generally considered a last resort. Queries that use NOLOCK aren't guaranteed to return correct results or technically, return any results at all.  

SQL 2005 has snapshot-based isolation levels that prevent readers from blocking writers or writers from blocking readers without allowing dirty reads.

Now, I have said that NOLOCK has served me personally very well in the past on systems of some size, but I hear what folks who say no to NOLOCK are saying. It certainly depends on one's definition of "correct results." ;)

There's three ways to get the behavior your want. Using TransactionScope is the recommended way to affect the transaction options associated with the commands generated by either LINQ to SQL or LINQ to Entities.

LINQ to SQL also supports explicitly setting the transaction on the context, so you could get the connection from the context, open it, start a transaction, and set it on the context. This can be desirable if you think SQL 2005 is promoting transactions too often, but the preferred method is TransactionScope.

ProductsNewViewData viewData = new ProductsNewViewData();
using (var t = new TransactionScope(TransactionScopeOption.Required,
    new TransactionOptions { 
IsolationLevel = System.Transactions.IsolationLevel.ReadUncommitted
})) { viewData.Suppliers = northwind.Suppliers.ToList(); viewData.Categories = northwind.Categories.ToList(); }

Here's an example where I used it in some recent code. This TransactionScope could be hidden (layered away) in your DAL (Data Access Layer) or in your Data Context directly if you wanted it to be neater.

A second way is that you can still create and call Stored Procedures (sprocs) from LINQ to SQL and those sprocs could include NOLOCK, TransactionScope is a better choice for LINQ to SQL or LINQ to Entity generated SQL if you feel that your query doesn't need to lock down the table(s) it's reading from.

Note that you'll want to be aware of which statement in your LINQ to SQL actually starts talking to the database. You can setup a query ahead of time and it won't be executed, for example, until someone calls ToList() or the like. It's at this point you'll want to wrap it in the using(TransactionScope){}.

Another third way you could set it at a DataContext level (which, to be clear, would affect every generated LINQ to SQL query executed on that context) would be to execute the command:

SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED

...using the connection available inside your DataContext.

Thanks to David Browne and Michael Pizzo for their help on this!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

The Weekly Source Code 19 - LINQ and More What, Less How

March 13, '08 Comments [19] Posted in Learning .NET | LINQ | Microsoft | Programming | Source Code
Sponsored By

Dear Reader, I present to you nineteenth in a infinite number of posts of "The Weekly Source Code."

At Mix, Clint Rutkas and I were messing around writing a plugin model for one of his apps. We were prototyping and I typed up this typical-looking plugin style code:

string[] filesToTest = Directory.GetFiles(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "plugins"), "*.dll");
foreach (string file in filesToTest)
{
    Assembly a = Assembly.LoadFrom(file);
    foreach (Type t in a.GetTypes())
    {
        if (!t.IsAbstract && t.BaseType == typeof(MyBaseClass))
        {
            myListOfInstances.Add((MyBaseClass)Activator.CreateInstance(t, credentials));
        }
    }
}

It's pretty straightforward and has little (read: no) error handling. It spins through the DLLs in the /plugins folder, loads them each (this should be more explicit rather than a blanket load of all dlls in a folder), then looks for any types that are derived from MyBaseClass that aren't abstract, then instantiates them and puts them in a list.

I'm going to quote Jeff Moser's very good blog post "What does it take to be a grandmaster" in both this post and the next one I write. It's that good. Seriously, go read it now, I'll wait here.

Jeff invokes Anders Heilsberg (Father of C# and LINQ) and:

"Anders' recent statement that future versions of C# will be about getting the programmer to declare "more of the what, and less of the how" with respect to how a result gets computed. A side effect of this is that your "chunks" tend to be more efficient for the runtime, and more importantly, your brain."

I'm going to repeat part I bolded. I like Programming Languages that allow me to declare "more of the what, and less of the how."  This how we tried to design the system at my last job and I automatically mentally migrate towards systems that encourage this kind of thinking.

So, back to the foreach loops above. My good friend (and fellow baby sign language fan) Kzu came by while Clint and I were working and suggested we turn this increasingly complex procedure into a LINQ query.

After a few tries, here's what we came up with.

myListOfInstances =
    (from file in Directory.GetFiles(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "plugins"), "*.dll")
     let a = Assembly.LoadFrom(file)
     from t in a.GetTypes()
     where !t.IsAbstract && t.BaseType == typeof(MyBaseClass) 
     select (MyBaseClass)Activator.CreateInstance(t, credentials))
    .ToList();

This is all one line, with some whitespace for readability. It says the same thing as the first chunk of code, but for some minds, it might be easier to read. This isn't a very good example of LINQ shining, from my point of view even though I like reading it. Certainly there's nowhere (I can see) for me to put error handling code that catches a bad load or bad cast in this example. There are, however, a LOT of places within that single line that one could set a breakpoint.

A better example would be like the one Jeff uses:

var primes = new int[] { 2,3,5,7,11,13,17,19,13,29,31,37,41 };
var primeSum = 0;
for (int i = 0; i < primes.Length; i++)
{
primeSum += primes[i];
}

...or even...

var primes = new int[] { 2,3,5,7,11,13,17,19,13,29,31,37,41 };
var primeSum = 0;
foreach (int i in primes)
{
  primeSum += i;
}

...could be more easily written and read like:

var primes = new int[] { 2,3,5,7,11,13,17,19,13,29,31,37,41 };
var primeSum = primes.Sum();

where .Sum() is part of LINQ as Arrays are IEnumerable.

I totally recommend you go get the free, no-installer LINQPad that comes with 200 examples from C# in a Nutshell.

Here are a few examples that I think really typify pretty code; certainly an improvement over the standard procedural code I'd usually write:

var names = new[] { "Tom", "Dick", "Harry" }.Where(n => n.Length >= 4);

Here's onen that does quite a few things to a list in a fairly clean syntax:

string[] names = { "Tom", "Dick", "Harry", "Mary", "Jay" };

var results =
     (
       from n in names
       where n.Contains("a")  // Filter elements
       orderby n.Length       // Sort elements
       select n.ToUpper()     // Translate each element (project)
     ).ToList();       	

What LINQ-style queries have you written that feel more what than how?

Related, ahem, LINQS

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb
Previous Page Page 2 of 5 in the LINQ category Next Page

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.