Scott Hanselman

The Weekly Source Code 15 - Tiny Managed Operating System Edition

February 4, '08 Comments [15] Posted in Learning .NET | Microsoft | Musings | Programming | Source Code
Sponsored By

Thanks to Thijs Kroesbergen for the pointer to this week's source. He turned me on to a tiny Operating System written in C# called "Cosmos (C# Open Source Managed Operating System)". As the project says, why? Because it's fun! I wrote a Tiny Virtual OS in C# for an Operating Systems class I took while I was going to school at night.

And so, Dear Reader, I present to you fifteenth in a infinite number of posts of "The Weekly Source Code." Here's some source I was reading this week.

First, I went back and looked at some of my source code from the Tiny OS project. It was written, largely at night, in .NET 1.1. It was for a 10 week (one term) project, and I wrote it all in a weekend so as to have the rest of the term free.

It's pretty horrific to read old source. I would propose that if you can look at old code you wrote and feel good about it that you're either insane, deranged or a really good programmer. One of those three.

Do remember that this project was a tiny Virtual OS. We were given an Instruction Set for our little processor and a format for programs and the goal was to write an OS to run them.

Here's the constructor for my MemoryManager. I'm sure doing a lot of work in the constructor! One of the fun little things I did for this Tiny OS was to swap memory pages as XML Files. It was a little joke because XML was such overkill and overhead for something so nuanced as a memory manager. I figured since I was simulating an OS's behavior using something as high-level as .NET, then why shouldn't I swap fake memory to disk as XML?

I wanted the source for this Tiny Virtual OS to ready like all the psuedo-code we'd been looking at in the Operating Systems books.

Looking back, I think it'd have been cool if I'd made a WinForms host and did a graphical view of memory that would allow folks to see things like memory fragmentation. Maybe I'll do one in WPF or as an XBAP, it'd be a cool learning tool for some of the students I talk to.

public MemoryManager(uint virtualMemSizeIn)
{
	// Find a size for addressableMemory that is on a page boundary
	virtualMemSize = CPU.UtilRoundToBoundary(virtualMemSizeIn, CPU.pageSize);

	// Size of memory must be a factor of CPU.pageSize
	// This was asserted when the CPU initialized memory
	uint physicalpages = (uint)(CPU.physicalMemory.Length/CPU.pageSize);
	uint addressablepages = (uint)(virtualMemSize/CPU.pageSize);
	
	_pageTable = new ArrayList((int)addressablepages);

	// Delete all our Swap Files
	foreach (string f in Directory.GetFiles(".","*.xml"))
		File.Delete(f);

	// For all off addressable memory...
	// Make the pages in physical and the pages that aren't in physical
	for (uint i = 0;i < virtualMemSize; i+=CPU.pageSize)
	{
		// Mark the Pages that are in physical memory as "false" or "not free"
		MemoryPage p;
		if (i < CPU.physicalMemory.Length) 
		{
			p = new MemoryPage(i, true);
			freePhysicalPages[(int)(i/CPU.pageSize)] = false;
		}
		else p = new MemoryPage(i, false);

		_pageTable.Add(p);
	}
}

Now my OS a trinket, to be clear. Cosmos, on the other hands, is darned interesting. How could you create a REAL OS (meaning an actual bootable OS off of hardware, be it virtual or physical, using IL? You translate the IL to ASM, of course. Very cool and darned clever.

Cosmos includes a compiler (IL2CPU, which is part of Cosmos) that reads the input file (usually the shell) and Cosmos libraries and compiles the resulting IL to x86 code. IL2CPU has a layer for cross platform and we plan to support other processors and platforms, including x64. IL2CPU also supports certain extension methods which allow C# code to interact directly with the CPU, registers, and ports in the kernel. IL2CPU contains some inline assembler, but there are no ASM files that need to be linked in.

Currently IL2CPU first outputs raw asm files (with IL comments) and then processes them through nasm (a free assembler). Later we plan to emit directly to binary.

The scenarios that Cosmos could be used in are very interesting. Because it's easy to write to and easy to build, you could create little mini-OSes with just the features you want. You could make an OS that just does DNS, or just does some REST service. Who knows. (Yes, I know you could also do a stripped down Linux). There is also talk about getting Cosmos to work on the Wii.

The example below is from Indy.IL2CPU.Assembler.X86.Native. As you can see, IL2CPU writes out ASM.

protected override void EmitDataSectionHeader(string aGroup, StreamWriter aOutputWriter) {
    base.EmitDataSectionHeader(aGroup, aOutputWriter);
    if (aGroup == MainGroup) {
          aOutputWriter.WriteLine("section .data");
          aOutputWriter.WriteLine("_start:  ");
          aOutputWriter.WriteLine("; multiboot header ");
          aOutputWriter.WriteLine("MBFLAGS equ 0x03 ; 4KB aligned modules etc., full memory info,  ");
          aOutputWriter.WriteLine("                        ; use special header (see below) ");
          aOutputWriter.WriteLine("dd 0x1BADB002           ; multiboot signature ");
          aOutputWriter.WriteLine("dd MBFLAGS              ; 4kb page aligment for modules, supply memory info ");
          aOutputWriter.WriteLine("dd -0x1BADB002-MBFLAGS  ; checksum=-(FLAGS+0x1BADB002) ");
          aOutputWriter.WriteLine("; other data - that is the additional (optional) header which helps to load  ");
          aOutputWriter.WriteLine("; the kernel. ");
          aOutputWriter.WriteLine("; end of header ");
          aOutputWriter.WriteLine("MultiBootInfo_Memory_High dd 0");
          aOutputWriter.WriteLine("MultiBootInfo_Memory_Low dd 0");
 }

There's lots of methods like this that do the hard work. The orchestration, however, is up in Engine.cs, where assemblies are taken appear via reflection, and their methods are taken apart using the obvious (my psuedo-code):

foreach Type in Assembly
   foreach Method in Type
      ProcessMethod into ASM (via ProcessAllMethods)

Using an interesting ILReader, a useful class in and of itself. Here's a trimmed chunk from ProcessAllMethods that uses the ILReader.

At this point, we've loaded an assembly, got a type, and we're sitting on xCurrentMethod. They take the method, check for some exception handling (I've trimmed that, it's tedious) and they get an OpCode, and then using the current Assembler, they assemble that operation. Lather, rinse, repeat as necessary.

ILReader xReader = new ILReader(xCurrentMethod);
    while (xReader.Read()) {
    if (mInstructionsToSkip > 0) {
        mInstructionsToSkip--;
        continue;
    }
    ExceptionHandlingClause xCurrentHandler = null;
    ...snip...        
    xMethodInfo.CurrentHandler = xCurrentHandler;
    xOp = GetOpFromType(mMap.GetOpForOpCode(xReader.OpCode), xReader, xMethodInfo);
    if ((!xOp.SupportsMetalMode) && mAssembler.InMetalMode) {
      throw new Exception("OpCode '" + xReader.OpCode + "' not supported in Metal mode!");
    }
    xOp.Assembler = mAssembler;
    new Comment("StackItems = " + 
      mAssembler.StackContents.Count + ", 
      Top item = " + 
       (mAssembler.StackContents.Count > 0 ? mAssembler.StackContents.Peek().ToString() : "(empty)"));
       xOp.Assemble();
    }

There's not a lot of comments in the Cosmos Project, but once you get your head around what they are doing, it's a pretty amazing piece of work, and I can see why they are having so much fun.

From their site at http://www.gocosmos.org:

If you just want to play with Cosmos:

  1. Install the user kit.
  2. Join the discussion list

Other resources:

  1. Read the FAQ.
  2. Subscribe to the Cosmos Blog
  3. Documentation - Most developer documentation right now.
If you are interested in kernel development:
  1. Get source from CodePlex
  2. Read the full requirements. They are pretty basic though, and everything you need except for Windows (For development) is free.
  3. Read Getting Started

Enjoy!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Monday, February 04, 2008 11:52:16 PM UTC
looks like fun ,Cosmos eh lets get dirty.BTW your tools set is an amazing archive ,so many useful tools :)
Donald
Monday, February 04, 2008 11:58:25 PM UTC
Ahah, I finally found the source of my vague memory of the guy who wrote a C# operating system for a class that used XML for something silly! It was you!


I would propose that if you can look at old code you wrote and feel good about it that you're either insane, deranged or a really good programmer.


Well, especially .NET 1.1 code... life before generics! All that boxing and unboxing!! So scary!

I'm starting to feel the same way about .NET 2.0 code now that I've started writing some fun LINQ stuff. Even many C# 3.0 language features, e.g. anonymous types, automatic properties, collection initializers, and of course lambdas, make my code feel so much happier.

Now if only they would implement anonymous collection initializers... (e.g. I want
from p in products select { { p.Name, p.Cost } }
.)
Tuesday, February 05, 2008 1:47:13 AM UTC
Ahhhh memories. Reading some of that takes me back to my operating systems class in college. We couldn't write our OS is a nice high level managed language. C with inline assembly is where it's at. malloc/free for the win!

Tuesday, February 05, 2008 1:50:49 AM UTC
How about the Singularity project? http://research.microsoft.com/os/singularity/
Tuesday, February 05, 2008 1:54:39 AM UTC
Just so you know, SharpOS has been around far longer than Cosmos.

www.sharpos.org
TraumaPony
Tuesday, February 05, 2008 2:52:11 AM UTC
"I would propose that if you can look at old code you wrote and feel good about it that you're either insane, deranged or a really good programmer."

How about: Dangerously complacent? There's no possible way a good developer looks at their old code and won't find flaws.
Tuesday, February 05, 2008 1:48:24 PM UTC
I'd have to agree with your comment about looking back at your old code. I pity the developer who has had to maintain some of the things I've wrote!
Thursday, February 07, 2008 2:10:41 AM UTC
This a totally wicked implementation. Nice Job!
Tuesday, February 26, 2008 8:44:25 AM UTC
Like:

public class PartyMan: Human , IHeadWithBrain, IEnumerable<IStraightHand>, IPerfectHealth
{
public Party party;
public Human FindContact()
{
foreach (Human cnt in party.Members)
if (cnt.Connectivity > 0.5)
{
this.WalkTo(cnt);
return cnt;
}
return NullMan;
}
public Drink(drinks Dr)
{
if (Dr == null)
Dr = BringDrink();
if (Dr.Openability)
Dr.Open();
while (this.Head.Eyes.IsHappenToMeet())
Dr.Sup();
}
... etc.
}
Vasiliy Zharkovskiy
Thursday, February 28, 2008 4:19:24 PM UTC
I am just wondering about the efficiency of the OS. Would an OS written in C# and then converted to ASM and then to binary would be as "fast and efficient" as an OS wriiten in C or ASM?

My first guess - no. I would like to hear the opinions of the other people.
Ashish K Mondal
Tuesday, March 04, 2008 8:56:24 AM UTC
I've heard surprisingly good things about C# with regards to speed. Perhaps it isn't as 'chainsaw'-ish as C or even C++, but it is certain far less likely to inadvertantly blow your own head off writing strings one character at a time.

There are some aspects of an operating system that I can't possibly imagine not requiring low-level code, though if we could leave the low-level code to the objects (out of sight out of mind), a higher-level understanding is not only possible, it's rather feasible as well.
Neil Harmon
Tuesday, March 04, 2008 3:45:24 PM UTC

At Sun, we implemented JavaOS back around 1996. Same thing, most of the OS was written in Java, with a small platform-dependent piece at the bottom written in C/assembly. This was for the JavaStation network computer. We had it running on Sparc and ARM processors. Since C#/CLR is basically a reimplementation of Java/JVM, this stuff is old hat. But hey, it is fun and interesting to play around with.
John
Wednesday, March 12, 2008 10:38:11 PM UTC

Since C#/CLR is basically a reimplementation of Java/JVM, this stuff is old hat


That so reminds me, as I used to tell people back in 1994, Java/JVM is basically a reimplementation of Visual Basic / VBRuntime.dll.

Never understood why the same people who told me that VB was bad because it wasn't a real language that was interpreted at runtime became such fans of Java when at it's heart it was exactly the same thing!
Matthew Hintzen
Thursday, March 13, 2008 9:04:54 AM UTC
Which is a re-implementation of the pascal p-code virtual machine...
Thursday, March 13, 2008 1:18:49 PM UTC
bah......
I was programming in 1984 using Basic on an ADAM (commodore), building screen savers, text to speech engines, etc.
Everything progresses. It's not about what is better, it's about need. As we've moved through the years, our needs have changed based on the requirements of users. Faster machines, more memory, bigger hard drives, more appealing UI's. Each language had and/or still has it's use.
Steve Nihan
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.