Scott Hanselman

The Weekly Source Code 15 - Tiny Managed Operating System Edition

February 4, '08 Comments [15] Posted in Learning .NET | Microsoft | Musings | Programming | Source Code
Sponsored By

Thanks to Thijs Kroesbergen for the pointer to this week's source. He turned me on to a tiny Operating System written in C# called "Cosmos (C# Open Source Managed Operating System)". As the project says, why? Because it's fun! I wrote a Tiny Virtual OS in C# for an Operating Systems class I took while I was going to school at night.

And so, Dear Reader, I present to you fifteenth in a infinite number of posts of "The Weekly Source Code." Here's some source I was reading this week.

First, I went back and looked at some of my source code from the Tiny OS project. It was written, largely at night, in .NET 1.1. It was for a 10 week (one term) project, and I wrote it all in a weekend so as to have the rest of the term free.

It's pretty horrific to read old source. I would propose that if you can look at old code you wrote and feel good about it that you're either insane, deranged or a really good programmer. One of those three.

Do remember that this project was a tiny Virtual OS. We were given an Instruction Set for our little processor and a format for programs and the goal was to write an OS to run them.

Here's the constructor for my MemoryManager. I'm sure doing a lot of work in the constructor! One of the fun little things I did for this Tiny OS was to swap memory pages as XML Files. It was a little joke because XML was such overkill and overhead for something so nuanced as a memory manager. I figured since I was simulating an OS's behavior using something as high-level as .NET, then why shouldn't I swap fake memory to disk as XML?

I wanted the source for this Tiny Virtual OS to ready like all the psuedo-code we'd been looking at in the Operating Systems books.

Looking back, I think it'd have been cool if I'd made a WinForms host and did a graphical view of memory that would allow folks to see things like memory fragmentation. Maybe I'll do one in WPF or as an XBAP, it'd be a cool learning tool for some of the students I talk to.

public MemoryManager(uint virtualMemSizeIn)
{
	// Find a size for addressableMemory that is on a page boundary
	virtualMemSize = CPU.UtilRoundToBoundary(virtualMemSizeIn, CPU.pageSize);

	// Size of memory must be a factor of CPU.pageSize
	// This was asserted when the CPU initialized memory
	uint physicalpages = (uint)(CPU.physicalMemory.Length/CPU.pageSize);
	uint addressablepages = (uint)(virtualMemSize/CPU.pageSize);
	
	_pageTable = new ArrayList((int)addressablepages);

	// Delete all our Swap Files
	foreach (string f in Directory.GetFiles(".","*.xml"))
		File.Delete(f);

	// For all off addressable memory...
	// Make the pages in physical and the pages that aren't in physical
	for (uint i = 0;i < virtualMemSize; i+=CPU.pageSize)
	{
		// Mark the Pages that are in physical memory as "false" or "not free"
		MemoryPage p;
		if (i < CPU.physicalMemory.Length) 
		{
			p = new MemoryPage(i, true);
			freePhysicalPages[(int)(i/CPU.pageSize)] = false;
		}
		else p = new MemoryPage(i, false);

		_pageTable.Add(p);
	}
}

Now my OS a trinket, to be clear. Cosmos, on the other hands, is darned interesting. How could you create a REAL OS (meaning an actual bootable OS off of hardware, be it virtual or physical, using IL? You translate the IL to ASM, of course. Very cool and darned clever.

Cosmos includes a compiler (IL2CPU, which is part of Cosmos) that reads the input file (usually the shell) and Cosmos libraries and compiles the resulting IL to x86 code. IL2CPU has a layer for cross platform and we plan to support other processors and platforms, including x64. IL2CPU also supports certain extension methods which allow C# code to interact directly with the CPU, registers, and ports in the kernel. IL2CPU contains some inline assembler, but there are no ASM files that need to be linked in.

Currently IL2CPU first outputs raw asm files (with IL comments) and then processes them through nasm (a free assembler). Later we plan to emit directly to binary.

The scenarios that Cosmos could be used in are very interesting. Because it's easy to write to and easy to build, you could create little mini-OSes with just the features you want. You could make an OS that just does DNS, or just does some REST service. Who knows. (Yes, I know you could also do a stripped down Linux). There is also talk about getting Cosmos to work on the Wii.

The example below is from Indy.IL2CPU.Assembler.X86.Native. As you can see, IL2CPU writes out ASM.

protected override void EmitDataSectionHeader(string aGroup, StreamWriter aOutputWriter) {
    base.EmitDataSectionHeader(aGroup, aOutputWriter);
    if (aGroup == MainGroup) {
          aOutputWriter.WriteLine("section .data");
          aOutputWriter.WriteLine("_start:  ");
          aOutputWriter.WriteLine("; multiboot header ");
          aOutputWriter.WriteLine("MBFLAGS equ 0x03 ; 4KB aligned modules etc., full memory info,  ");
          aOutputWriter.WriteLine("                        ; use special header (see below) ");
          aOutputWriter.WriteLine("dd 0x1BADB002           ; multiboot signature ");
          aOutputWriter.WriteLine("dd MBFLAGS              ; 4kb page aligment for modules, supply memory info ");
          aOutputWriter.WriteLine("dd -0x1BADB002-MBFLAGS  ; checksum=-(FLAGS+0x1BADB002) ");
          aOutputWriter.WriteLine("; other data - that is the additional (optional) header which helps to load  ");
          aOutputWriter.WriteLine("; the kernel. ");
          aOutputWriter.WriteLine("; end of header ");
          aOutputWriter.WriteLine("MultiBootInfo_Memory_High dd 0");
          aOutputWriter.WriteLine("MultiBootInfo_Memory_Low dd 0");
 }

There's lots of methods like this that do the hard work. The orchestration, however, is up in Engine.cs, where assemblies are taken appear via reflection, and their methods are taken apart using the obvious (my psuedo-code):

foreach Type in Assembly
   foreach Method in Type
      ProcessMethod into ASM (via ProcessAllMethods)

Using an interesting ILReader, a useful class in and of itself. Here's a trimmed chunk from ProcessAllMethods that uses the ILReader.

At this point, we've loaded an assembly, got a type, and we're sitting on xCurrentMethod. They take the method, check for some exception handling (I've trimmed that, it's tedious) and they get an OpCode, and then using the current Assembler, they assemble that operation. Lather, rinse, repeat as necessary.

ILReader xReader = new ILReader(xCurrentMethod);
    while (xReader.Read()) {
    if (mInstructionsToSkip > 0) {
        mInstructionsToSkip--;
        continue;
    }
    ExceptionHandlingClause xCurrentHandler = null;
    ...snip...        
    xMethodInfo.CurrentHandler = xCurrentHandler;
    xOp = GetOpFromType(mMap.GetOpForOpCode(xReader.OpCode), xReader, xMethodInfo);
    if ((!xOp.SupportsMetalMode) && mAssembler.InMetalMode) {
      throw new Exception("OpCode '" + xReader.OpCode + "' not supported in Metal mode!");
    }
    xOp.Assembler = mAssembler;
    new Comment("StackItems = " + 
      mAssembler.StackContents.Count + ", 
      Top item = " + 
       (mAssembler.StackContents.Count > 0 ? mAssembler.StackContents.Peek().ToString() : "(empty)"));
       xOp.Assemble();
    }

There's not a lot of comments in the Cosmos Project, but once you get your head around what they are doing, it's a pretty amazing piece of work, and I can see why they are having so much fun.

From their site at http://www.gocosmos.org:

If you just want to play with Cosmos:

  1. Install the user kit.
  2. Join the discussion list

Other resources:

  1. Read the FAQ.
  2. Subscribe to the Cosmos Blog
  3. Documentation - Most developer documentation right now.
If you are interested in kernel development:
  1. Get source from CodePlex
  2. Read the full requirements. They are pretty basic though, and everything you need except for Windows (For development) is free.
  3. Read Getting Started

Enjoy!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

MS Deploy - New IIS Web Deployment Tool

February 2, '08 Comments [12] Posted in ASP.NET | IIS | Tools
Sponsored By

Microsoft Web Deployment Agent Service Properties (Local Computer)Months ago, even before I started at Microsoft, I got to talk to some very nice people about a project they were calling AdminX. After I told them that that was very possibly the worst name they could possibly give it (and they took part that very well) we got to dig into what the tool was going to do. As with all great tools it did stuff that we had already written at my last company. It's always nice to throw home-grown tools away if found tools will do the job better.

Anyway, there's a Tech Preview of MS Deploy out for both x86 and x64. It installs a Windows Service, but it doesn't need to be running except when it's doing work so don't panic.

It's got a metric ton (a good thing) of options. For example, if I want to archive/backup a Site, I go:

msdeploy.exe" -verb:sync -source:appHostConfig="Default Web Site" -dest:archivedir=c:\mybackup

You can restore just by reversing source and dest. All the settings are maintained and stored in XML.

There's other "verbs" to use. For example, you can getDependencies (currently only on IIS6, IIS7 to come) and get details on exactly what your app needs in terms of authentication, filters, modules, etc.

You can transfer or 'sync' websites or web servers between each other like this:

msdeploy -verb:sync -source:webserver,targetName=http://computername/MSDEPLOY/ -dest:webserver > msdeploysync.log

Also, because there's a Windows Service involved, you can do remote deploys - that's the part we've all been waiting for. MS Deploy also supports the -whatif flag, showing you what will happen without doing anything. 

There is also a plan to have a PowerShell cmdlet interface for MS Deploy when it releases, so things are looking up in the IIS space.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Firefox, WPF and XBAP

February 2, '08 Comments [27] Posted in Microsoft | Programming | Windows Client
Sponsored By

WoodGrove Finance Explorer - Mozilla FirefoxI finally got around to trying a .NET 3.5 XBAP (XAML Browser App or "WPF Browser Apps") in Firefox, and it works just as advertised. I put together an incredibly useful and powerful application consisting of a picture of me and a button. I think everyone will want to run it. ;)

Seriously though, it's very easy to deploy apps like this. This reminds me of the year I spent working for Aurum (then a division of Baan) creating a large application using VBPs - ActiveX Documents.

These were the same basic idea. Your application would run - then IE4, methinks - using the Browser as it's Window Frame, much the way Word or Acrobat can open up a document inside the Browser. This is all still very old-school ISite COM stuff.

Anyway, XBAPs aren't Silverlight, they are the Full .NET Framework in your browser. With .NET 3.5 that means IE or Firefox. Think of XBAPs as ClickOnce applications that never jump out of the browser.

Keep in mind that mine is a silly example, and yes, this one could be done with DHTML, however the Woodgrove Finance Application (a .NET 3.0 WPF Application, seen above in Firefox) would be more challenging, hence the idea behind WPF Browser Apps.

I fire up VS2008 and hit File | New Project | WPF Browser Application.

image

Then, I drag an image on the Page and a Button. I use Split Screen View so I can see the XAML being written a the same time. I double-click on the surface, then go back and double-click on the button. That adds the Loaded= and Click= event handlers you see below. I could have typed this manually also.

<Page x:Class="WpfBrowserApplication1.Page1"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    Title="Page1" Loaded="Page_Loaded">
    <Grid>
        <Button Height="57" Margin="70,0,70,24" Name="button1" 
VerticalAlignment="Bottom" Click="button1_Click">What a great App!</Button> <Image Name="image1" Margin="50,18,48,96" > <Image.RenderTransform> <RotateTransform Angle="0" CenterX="100" CenterY="100" /> </Image.RenderTransform> </Image> </Grid> </Page>

I'm going to do two things. One, I'll load a picture from an Embedded Resource. I could have loaded it from a location like my web server, but this is harder and educational for me. Then, two, I'll make the picture turn when I push the button. I could add an Animation declaratively but again, this is a little harder and more interesting.

First, getting the embedded graphical resource. This might look familiar if you've done it with WinForms.

System.IO.Stream stream = this.GetType().Assembly.
GetManifestResourceStream("WpfBrowserApplication1.MyFile.jpg"); JpegBitmapDecoder bitmapDecoder =
new JpegBitmapDecoder(stream,
BitmapCreateOptions.PreservePixelFormat,
BitmapCacheOption.Default); ImageSource imageSource = bitmapDecoder.Frames[0]; image1.Source = imageSource;

I've made it more lines than is needed, but basically, get the stream, decode the graphic (Gif, Jpeg, PNG) and grab the Frame. If it were a Gif, it might be animated, hence Frames[0].

Ok, easy. Now to make it spin. And not just turn in a chunky way, but smoothly turn 60 degrees with each button press. I make an animation, set a From and To value as a double and tell it to last a half second. Then I grab the RotateTransform off the image and begin the animation.

private int angle = 0;
private void button1_Click(object sender, RoutedEventArgs e)
{
  DoubleAnimation ani = new DoubleAnimation();
  ani.From = angle;
  angle += 60;
  ani.To = angle;
  if (angle >= 360) angle = 0;
  ani.Duration = new Duration(new TimeSpan(0, 0, 0, 0, 500));

  RotateTransform tran = (RotateTransform)image1.RenderTransform;
  tran.BeginAnimation(RotateTransform.AngleProperty, ani);
}

There's a pile of good articles and books out there on WPF. What strikes me is how many high level it is. It's nice to think of constructs like angle and RotateTransform rather than doing the math.

The silly result of this code is at http://www.hanselman.com/clickonce/takethree/WpfBrowserApplication1.xbap and it works in both Firefox and IE if you have .NET 3.5 installed.

Related Posts

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

First Half 2008 Conference Speaking Schedule

January 31, '08 Comments [28] Posted in ASP.NET | Speaking
Sponsored By

image Well, I think I've got my schedule down for the first half of this year. Here's what I've got so far.

I'm trying to work out a way to get over to the Middle East to see and talk to Developers over there, unfortunately the flights from here usually take between 21 and 29 hours, as opposed to less than 10 to use Norway as an example. This is really hard on my diabetes (and Norway will be also but it's 1/3 as long) but it's also hard on the family and I need to keep balance. That said, I'm actively working on a way to hang out in EMEA for some time, maybe bring the family along.

I hope I'll see some of you at one of these events!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

The Weekly Source Code 14 - Fluent Interface Edition

January 31, '08 Comments [20] Posted in ASP.NET | Learning .NET | Programming | Ruby | Source Code
Sponsored By

iStock_000000237891XSmall5 If you're new to this, each week I post some snippets of particularly interesting (read: beautiful, ugly, clever, obscene) source and the project it came from. This started from a belief that reading source is as important (or more so) as writing it. We read computer books to become better programmers, but unless you're reading books like Programming Pearls, you ought to peruse some Open Source projects for inspiration.

And so, Dear Reader, I present to you fourteenth in a infinite number of posts of "The Weekly Source Code." Here's some source I was reading this week.

Over the last year or so I've seen an increase in discussion around so-called "fluent interfaces" in many languages. The addition of extension methods (mixins) to C# 3.0 has caused a flood of interesting (weird?) interface as we as a collective to attempt to make programming as easy as writing prose.

Martin Fowler talked about this in 2005 after a workshop with Eric Evans when they first named them "fluent interfaces." He gives this example:

The simplest example is probably from Eric's timeAndMoney library. To make a time interval in the usual way we might see something like this:

TimePoint fiveOClock, sixOClock;
...
TimeInterval meetingTime = new TimeInterval(fiveOClock, sixOClock);

The timeAndMoney library user would do it this way:

   TimeInterval meetingTime = fiveOClock.until(sixOClock);

Of course, the ubiquitous Ruby example is

20.minutes.ago

Martin makes a great point when trying to put "fluent" APIs in a common OOP context, like that of an object browser with emphasis mine:

One of the problems of methods in a fluent interface is that they don't make much sense on their own. Looking at a method browser of method by method documentation doesn't show much sense to with. Indeed sitting there on its own I'd argue that it's a badly named method that doesn't communicate its intent at all well. It's only in the context of the fluent action that it shows its strengths. One way around this may be to use builder objects that are only used in this context. - Martin Fowler

Piers Cawley follows up and offers a number of guidelines for use when one is designing these things. See his post for the complete annotated list.

  1. Hide your working.
  2. Keep your state to yourself.
  3. Think really hard about names.
  4. Take advantage of your implementation language.
  5. If you have them, blocks are you friends.
  6. Test first design can be a useful way of exploring what your interface should be.
  7. Reasonable defaults.

In the .NET space, Ayende's Rhino Mocks are, I think, the first and best example before LINQ that really got it right with syntax like.

  Expect
.Call(mock.GetID(1))
.IgnoreArguments()
.Repeat
.Once()
.Return(something);

Similar things are done in Java with their support for mixins, called Static Imports in Java 5.

When fluent interfaces get larger and more complex, they suddenly get called Domain Specific Languages as Peirs points out. But, a true DSL is even easier and might not be fluent at all, but rather customized to the domain:

"It seems that every time someone writes a Ruby library that uses class methods, symbols and hashes reasonably sensibly they get delusions of grandeur and call the result a Domain Specific Language (or maybe an ‘embedded’ DSL)."

Two good examples of libraries as DSLs with some fluent aspects are Why's Hpricot, an HTML Parser for Ruby that looks like this:

 #!ruby
 require 'hpricot'
 require 'open-uri'
 # load the RedHanded home page
 doc = Hpricot(open("http://redhanded.hobix.com/index.html"))
 # change the CSS class on links
 (doc/"span.entryPermalink").set("class", "newLinks")
 # remove the sidebar
 (doc/"#sidebar").remove
 # print the altered HTML
 puts doc

And Python's Beautiful Soup, also an HTML Parser.

 from BeautifulSoup import BeautifulSoup, Tag
 soup = BeautifulSoup("Argh!FooBlah!")
 tag = Tag(soup, "newTag", [("id", 1)])
 tag.insert(0, "Hooray!")
 soup.a.replaceWith(tag)
 print soup
 # Argh!Hooray!Blah!

Back on the C# side, Garry Shutler is creating more fluent assertions using extension methods and lambdas for MBUnit like:

testObject.ShouldBeTheSameObjectAs(targetObject).And.ShouldBeEqualTo(testObject).And.ShouldSatisfy(x => x is Object);

But what's a DSL and what's a Fluent Interface and what's just an API? Martin adds in 2006 (as he continues to write his DSL Book):

For me, a key element is that DSLs are limited both in scope (they refer to a particular domain) and capability (they lack features that are basic for general purpose languages). As a result good DSLs are usually small and simple: hence terms like 'little languages' and 'mini-languages'.

For internal DSLs, the fuzzy boundary is what is an API and what is a DSL. Fundamentally there is no difference, an internal DSL is just an API with a fancy name (as the old Bell labs saying goes: "library design is language design"). Despite this, however, I think there is a different feel when you are working with an API that's written with a DSL feel. Things like a FluentInterface can make working with an API a qualitatively different experience. Thinking in DSL terms makes you think about readability in a different way, exploiting the syntax of the host language to create something that seems to stand on its own - rake is a great example of this. - Martin Fowler

Even scripting languages like PHP are getting on board with fluent interfaces, assuming that "with" in this context makes sense to you.

<?php
private function makeFluent(Customer $customer) {
    $customer->  newOrder()
         ->with(6, 'TAL')
         ->with(5, 'HPK')->skippable()
         ->with(3, 'LGV')
         ->priorityRush();           

Ultimately I think Paul Jones nails it when he says "Fluent Interfaces Require Fluent Situations":

"I think, for a fluent interface to be effective, you need situations where you actually have all that information at one time so that you can chain the methods in a fluid way" - Paul M. Jones

Scott Bellware said matter of factly:

"Whether fluent interface is a form of DSL or not, it's obviously a form of fluent interface." - Scott Bellware

Are you exploring Fluent Interfaces?


Shameless Plug: As an aside, one addition thing I'd like to mention is our little Forums over at http://hanselman.com/forum. They are run on excellent JitBit's AspNetForum application, the "little forum that could" in my opinion. There's lots of interesting discussions on many diverse topics, and you can look at just the most recent posts, and every page has an RSS Feed.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.