Scott Hanselman

I've had a number of great guests on the podcast lately. One topic that has come up a number of times is the "toy project." I've usually kept mine private - never putting them on GitHub - Somewhat concerned that people would judge me and my code. However, hypocrite that am (aren't we all?) I have advocated that others put their "Garage Sale Code" online. So here's some crappy code. ;)

The Preamble

While I've been working as an engineer for 25 years this year, I didn't graduate from school with a 4 year degree until 2003 - I just needed to get it done, for myself. I was poking around recently and found my project from OIT's CST352 "Operating Systems" class. One of the projects was to create a "Virtual CPU and OS." This is kind of a thought exercise. It's not really a parser/lexer - although there is both - and it's not a real OS. But it needs to be able to take in a made-up quasi-Assembly Language instruction set and execute them on a virtual CPU while managing virtual memory of arbitrary size. Again, a thought exercise made real to confirm that the student understands the responsibilities of a CPU.

Here's an example "application." Confused yet? Here's the original spec I was given in 2002 that includes the 36 instructions the "CPU" should understand. It has 10 general-purpose 32bit registers address as 1 through 10. Register 10 is the stack pointer. There are two bit flag registers - sign flag and zero flag.

Instructions are "opcode arg1 arg2" with constants prefixed with "$."

11 r8        ;Print r8
6  r1 $10    ;Move 10 into r1
6  r2 $6     ;Move 6 into r2
6  r3 $25    ;Move 25 into r3
23 r1        ;Acquire lock in r1 (currently 10)
11 r3        ;Print r3 (currently 25)
24 r1        ;Release r1 (currently 10)
25 r3        ;Sleep r3 (currently 25)
11 r3        ;Print r3 (currently 25)
27           ;Exit

I write my homework assignment in 2002 in the idiomatic C# of the time on .NET 1.1. That means no Generics<T> - I had to make my own strongly typed collections. That means C# has dozens of (if not a hundred) language and syntax improvements. I didn't use a Unit Testing Framework as TDD was just starting around 1999 during the XP (eXtreme Programming) days and NUnit was just getting start. It also uses "unsafe" to pin down memory in a few places. I'm sure there are WAY WAY WAY better and more sophisticated ways to do this today in idiomatic C# of 2017. Those are excuses, the real reasons are my own ignorance, ability, combined with some night-school laziness.

One of the more fun parts of this exercise was moving from physical memory (a byte array as I recall) to a full-on Memory Manager where each Process thought it could address a whole bunch of Virtual Memory while actual Physical Memory was arbitrarily sized. Then - as a joke - I would swap out memory pages as XML! ;) Yes, to be clear, it was a joke and I still love it.

You can run an "app" by passing in the total physical memory along with the text file containing the program, but you can also run an arbitrary number of programs by passing in an arbitrary number  of text files! The "TinyOS" will handle each process thinking it has its own memory and will time

If you are more of a visual learner, perhaps you'd prefer this 20-slide PowerPoint on this Tiny CPU that I presented in Malaysia later that year. You dig those early 2000-era slides? I KNOW YOU DO.

Updating a .NET 1.1 app to cross-platform .NET Core 2.0

Step 1 was to download the original code from my own blog. ;) This is also Reason #4134 why you should have a blog.

I decided to use Visual Studio 2017 to upgrade it, and even worse I decided to use .NET Core 2.0 which is currently in Preview. I wanted to use .NET Core 2.0 not just because it's cross-platform but also because it promises to have a pretty large API surface area and I want this to "just work." The part about getting my old application running on Linux is going to be awesome, though.

Visual Studio then pops a scary dialog about upgrading files. NOTE that another totally valid way to do this (that I will end up doing later in this blog post) is to just make a new project and move the source files into it. Natch.

Visual Studio says it's targeting .NET 2.0 Full Framework, but I ratchet it up to 4.6 to see what happens. It builds but with a bunch of errors about Obsolete methods, the most interesting one being this one:

Warning CS0618    
'ConfigurationSettings.AppSettings' is obsolete: 
'This method is obsolete, it has been replaced by 
System.Configuration!System.Configuration.ConfigurationManager.AppSettings'    
C:\Users\scott\Downloads\TinyOSOLDOLD\OS Project\CPU.cs    72

That's telling me that my .NET 1/2 API will work but has been replaced in .NET 4.x, but I'm more interested in .NET Core 2.0. I could make my EXE a LIB and target .NET Standard 2.0 or I could make a .NET Core 2.0 app and perhaps get a few more APIs. I didn't do a formal analysis with the .NET Portability Analyzer but I will add that to the list of Things To Do. I may be able to make a library that works on an iPhone - a product that didn't exist when I started this assignment. That would be Just Cool(tm).

I decided to just make a new empty .NET Core 2.0 app and copy the source .cs files into it. A few interesting things.

• My app also used "unsafe" code (it pins memory down and accesses it directly).
• It has extensive inline documentation in comments that I used to use NDoc to make a CHM Help file. I'd like that doc to turn into HTML at some point.
• It also has an appsettings.json file that needs to get copied to the output folder when it compiles.
• While I could publish it to a self-contained .NET Core exe, for now I'm running it like this in my test batch files - example:
  • dotnet netcoreapp2.0/TinyOSCore.dll 512 scott13.txt

Here's the resulting csproj file.

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp2.0</TargetFramework>
    <GenerateDocumentationFile>true</GenerateDocumentationFile>
  </PropertyGroup>

  <PropertyGroup>
    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
  </PropertyGroup>

  <ItemGroup>
    <None Remove="appsettings.json" />
  </ItemGroup>

  <ItemGroup>
    <Content Include="appsettings.json">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </Content>
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.Extensions.Configuration" Version="2.0.0-preview2-final" />
    <PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="2.0.0-preview2-final" />
    <PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="2.0.0-preview2-final" />
    <PackageReference Include="Microsoft.Extensions.Options.ConfigurationExtensions" Version="2.0.0-preview2-final" />
  </ItemGroup>

</Project>

Other than the obsolete configuration warning and a few malformed XML comments, the app compiled and ran! You can actually "watch" the nightmare process here https://github.com/shanselman/TinyOS/commits/Core2Port in the form of GitHub commits. I also moved the docs from a 2002 Word Doc to Markdown so be sure to explore the fairly extensive spec https://github.com/shanselman/TinyOS.

The only significant change was loading the config. Configuration is even more different on .NET Core 2.0 than Full Framework. It's FAR more, ahem, configurable. I could have used "Options," I could have written my own config provider if it was important to keep the file format.

This little TinyOS has a bunch of config options that come in from a .exe.config file in XML like this (truncated):

<configuration>
    <appSettings>
    <!-- 
        Must be a factor of 4 
        This is the total Physical Memory in bytes that the CPU can address.  
        This should not be confused with the amount of total or addressable memory 
        that is passed in on the command line.
      -->
    <add key="PhysicalMemory" value="128" />
    <!-- 
        Must be a factor of 4 
        This is the ammount of memory in bytes each process is allocated
        Therefore, if this is 256 and you want to load 4 processes into the OS,
        you'll need to pass a number > 1024 as the total ammount of addressable memory 
        on the command line.
      -->
    <add key="ProcessMemory" value="384" />
    <add key="DumpPhysicalMemory" value="true" />
    <add key="DumpInstruction" value="true" />
    <add key="DumpRegisters" value="true" />
    <add key="DumpProgram" value="true" />
    <add key="DumpContextSwitch" value="true" />
    <add key="PauseOnExit" value="false" />

I have a few choices. I could make a Configuration Provider and reach .NET Core to read this format (there's an XML adapter, in fact) or make the code porting easier by moving these "name/value" pairs to a JSON file like this:

{
        "PhysicalMemory": "128",
        "ProcessMemory": "384",
        "DumpPhysicalMemory": "true",
        "DumpInstruction": "true",
        "DumpRegisters": "true",
        "DumpProgram": "true",
        "DumpContextSwitch": "true",
        "PauseOnExit": "false",
        "SharedMemoryRegionSize": "16",
        "NumOfSharedMemoryRegions": "4",
        "MemoryPageSize": "16",
        "StackSize": "16",
        "DataSize": "16"
}

This was just a few minutes of search and replace to change the XML to JSON. I could have also written a little app or shell script. By changing the config (rather than writing an adapter) I could then keep the code 99% the same.

My code was doing things like this (all over...there was no DI container yet):

bytesOfPhysicalMemory = uint.Parse(ConfigurationSettings.AppSettings["PhysicalMemory"]);

And I'd like to avoid major refactoring - yet. I added this bit of .NET Core configuration at the top of the EntryPoint and saved away an IConfigurationHost:

var builder = new ConfigurationBuilder()
    .AddJsonFile("appsettings.json");
Configuration = builder.Build();

I've got a Dictionary in the format of the IConfiguration host called "Configuration." So now I just do this in a dozen places and the app compiles again:

bytesOfPhysicalMemory = uint.Parse(Configuration["PhysicalMemory"]);

This brings up that feeling we all have when we look at old code - especially our own old code. I should have abstracted that away! Why didn't I use an interface? Why so many statics? What was I thinking?

We can beat ourselves up or we can feel good about ourselves and remember this. The app worked. It still works. There is value in it. I learned a lot. I'm a better programmer now. I don't know how far I'll take this old code but I had a lovely afternoon porting it to .NET Core 2.0 and I may refactor the heck out if it or I may not.

For now I did update the smoke tests to run on both Windows and Linux and I'm happy with the experiment.

Related Links

• Download PPT Slides on the Tiny OS presented at TechEd Malaysia 2002
• Andy Clarke from New Zealand took the original spec and did the homework assignment in 2012! His project - 5 years ago, and 10 years after mine - includes some interesting changes. Rather than an EXE that takes in the programs from the command line, he's written over 221 NUnit 2 tests that check each individual component he's written as well as more comprehensive integration tests (as unit test) for the programs. The "Assembly" language has been changed from opcodes to more human readable commands like "move" and "add." I think Andy's solution is much nicer than mine, but he wouldn't pass the class because the spec was pretty clear and my teacher was a stickler. ;) I LOVE that someone else did this on their own!
• PODCAST: YOU should write an interpreter with Thorsten Ball

Have YOU done a project like this, either in school or on your own?

Sponsor: Check out JetBrains Rider: a new cross-platform .NET IDE. Edit, refactor, test, build and debug ASP.NET, .NET Framework, .NET Core, or Unity applications. Learn more and get access to early builds!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Hosting By

The .NET Core team really prides themselves on performance. However, it's not immediately obvious (as with all systems) if you just do Hello World as a developer. Just today I was doing a Ruby on Rails app in Development Mode with mruby - but that's not what you'd go to production with.

Let's look at a great question I got today on Twitter.

Dotnet Run - Builds and Runs Source Code in Development

That's a great question. If you install .NET Core 2.0 Preview - this person is on a Mac, but you can use Linux or Windows as well - then do just this:

$ dotnet new console
$ dotnet run

It'll be about 3-4 seconds. dotnet is the SDK and dotnet run will build and run your source code. Here's a short bit from the docs:

The dotnet run command provides a convenient option to run your application from the source code with one command. It's useful for fast iterative development from the command line. The command depends on the dotnet build command to build the code. Any requirements for the build, such as that the project must be restored first, apply to dotnet run as well.

While this is super convenient, it's not totally obvious that dotnet run isn't something you'd go to production with (especially Hello World Production, which is quite demanding! ;) ).

Dotnet Publish then Dotnet YOUR.DLL for Production

Instead, do a dotnet publish, note the compiled DLL created, then run "dotnet tst.dll."

For example:

C:\Users\scott\Desktop\tst> dotnet publish
Microsoft (R) Build Engine version 15.3 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  tst -> C:\Users\scott\Desktop\tst\bin\Debug\netcoreapp2.0\tst.dll
  tst -> C:\Users\scott\Desktop\tst\bin\Debug\netcoreapp2.0\publish\
C:\Users\scott\Desktop\tst> dotnet run .\bin\Debug\netcoreapp2.0\tst.dll
Hello World!

On my machine, dotnet run is 2.7s, but dotnet tst.dll is 0.04s.

Dotnet publish --self-contained

I could then publish a complete self-contained app - I'm using Windows, so I'll publish for Windows but you could even build on a Windows machine but target a Mac runtime, etc and that will make a \publish folder.

C:\Users\scott\Desktop\tst> dotnet publish  --self-contained -r win10-x64
Microsoft (R) Build Engine version 15.3 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  tst -> C:\Users\scott\Desktop\tst\bin\Debug\netcoreapp2.0\win10-x64\tst.dll
  tst -> C:\Users\scott\Desktop\tst\bin\Debug\netcoreapp2.0\win10-x64\publish\
C:\Users\scott\Desktop\tst> .\bin\Debug\netcoreapp2.0\win10-x64\publish\tst.exe
Hello World!

Note in this case I have a "Self-Contained" app, so all of .NET Core is in that folder and below. Here I run tst.exe, not dotnet.exe because now I'm an end-user.

I hope this helps clear things up.

Sponsor: Check out JetBrains Rider: a new cross-platform .NET IDE. Edit, refactor, test, build and debug ASP.NET, .NET Framework, .NET Core, or Unity applications. Learn more and get access to early builds!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Hosting By

There's a ton of cool new .NET Core open source projects lately, and I've very much enjoyed exploring this rapidly growing space. Today at lunch I was checking out a project called "Brighter." It's actually been around in the .NET space for many years and is in the process of moving to .NET Core for greater portability and performance.

Brighter is a ".NET Command Dispatcher, with Command Processor features for QoS (like Timeout, Retry, and Circuit Breaker), and support for Task Queues"

Whoa, that's a lot of cool and fancy words. What's it mean? The Brighter project is up on GitHub incudes a bunch of libraries and examples that you can pull in to support CQRS architectural styles in .NET. CQRS stands for Command Query Responsibility Segregation. As Martin Fowler says, "At its heart is the notion that you can use a different model to update information than the model you use to read information." The Query Model reads and the Command Model updates/validates. Greg Young gives the first example of CQRS here. If you are a visual learner, there's a video from late 2015 where Ian Cooper explains a lot of this a the London .NET User Group or an interview with Ian Cooper on Channel 9.

Brighter also supports "Distributed Task Queues" which you can use to improve performance when you're using a query or integrating with microservices.

When building distributed systems, Hello World is NOT the use case. BUT, it is a valid example in that it strips aside any business logic and shows you the basic structure and concepts.

Let's say there's a command you want to send. The GreetingCommand. A command can be any write or "do this" type command.

internal class GreetingCommand : Command
{
    public GreetingCommand(string name)
        :base(new Guid())
    {
        Name = name;
    }

    public string Name { get; private set; }
}

Now let's say that something else will "handle" these commands. This is the DoIt() method. No where do we call Handle() ourselves. Similar to dependency injection, we won't be in the business of calling Handle() ourselves; the underlying framework will abstract that away.

internal class GreetingCommandHandler : RequestHandler<GreetingCommand>
{
    [RequestLogging(step: 1, timing: HandlerTiming.Before)]
    public override GreetingCommand Handle(GreetingCommand command)
    {
        Console.WriteLine("Hello {0}", command.Name);
        return base.Handle(command);
    }
}

We then register a factory that takes types and returns handlers. In a real system you'd use IoC (Inversion of Control) dependency injection for this mapping as well.

Our Main() has a registry that we pass into a larger pipeline where we can set policy for processing commands. This pattern may feel familiar with "Builders" and "Handlers."

private static void Main(string[] args)
{
   var registry = new SubscriberRegistry();
   registry.Register<GreetingCommand, GreetingCommandHandler>();


   var builder = CommandProcessorBuilder.With()
       .Handlers(new HandlerConfiguration(
            subscriberRegistry: registry,
            handlerFactory: new SimpleHandlerFactory()
           ))
       .DefaultPolicy()
       .NoTaskQueues()
       .RequestContextFactory(new InMemoryRequestContextFactory());

   var commandProcessor = builder.Build();

   ...
}

Once we have a commandProcessor, we can Send commands to it easier and the work will get done. Again, how you ultimately make the commands is up to you.

commandProcessor.Send(new GreetingCommand("HanselCQRS"));

Methods within RequestHandlers can also have other behaviors associated with them, as in the case of "[RequestLogging] on the Handle() method above. You can add other stuff like Validation, Retries, or Circuit Breakers. The idea is that Brighter offers a pipeline of handlers that can all operate on a Command. The Celery Project is a similar project except written in Python. The Brighter project has stated they have lofty goals, intending to one day handle fault tolerance like Netflix's Hystrix project.

One of the nicest aspects to Brighter is that it's prescriptive but not heavy-handed. They say:

Brighter is intended to be a library not a framework, so it is consciously lightweight and divided into packages that allow you to consume only those facilities that you need in your project.

Moving beyond Hello World, there are more fleshed out examples like a TaskList with a UI, back end Http API, a Mailer service, and core library.

Be sure to explore Brighter's excellent documentation and examples, but be aware, this is a project under active development. Perhaps if you're new to OSS, if you find a broken link or two or a misspelling, you can do Your First Pull Request with a small fix?

Do be aware, again, that CQRS is not for every project. It's non-trivial and it's a "mental leap" as Martin Fowler puts it. If you buy in, you're adding complexity...for a reason. Keep your eyes open and do your research. It's a great pattern if you have a high performance/volume application that struggles with write concurrency or a flaky backend.

In fact there are quite a few mature CQRS libraries in the .NET open source space. I'll explore a few - which are your favorites?

Sponsor: Seq is simple centralized logging, on your infrastructure, with great support for ASP.NET Core and Serilog. Version 4 adds integrated dashboards and alerts - check it out!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Hosting By

I've got a personal Surface Pro 3 that I like very much. It's worked great for years and I haven't had any issues with it. However, yesterday while installing a 3rd party USB device something got goofed around with the drivers and I ended up in this state.

That "banged out" device in my Device Manager is the root Universal Serial Bus (USB) Controller for the Surface. That means everything  USB didn't work since everything USB hangs off that root device node. I know it's an Intel USB 3.0 xHCI Host Controller but I didn't want to go installing random Intel Drivers. I just wanted the Surface back the way it was, working, with the standard drivers.

I tried the usual stuff like Uninstalling the Device and rebooting, hoping Windows would heal it but it didn't work. Because the main USB device was dead that meant my Surface Type Keyboard didn't work, my mouse didn't work, nothing. I had to do everything with the touchscreen.

After a little poking around on Microsoft Support websites, a friend turned me onto the "Surface Tools for IT." These are the tools that IT Departments use when they are rolling out a bunch of Surfaces to an organization and they are regularly updated. In fact, these were updated just yesterday!

There are a number of utilities you can check out but the most useful is the Surface Diagnostic Toolkit. It checks hardware and software versions and found a number of little drivers things wrong...and fixed them. It reset my USB Controller and put in the right driver and I'm back in business.

This util was useful enough to me that I wish it had been installed by default on the Surface and plugged into the built-in Windows Troubleshooting feature.

Sponsor: Seq is simple centralized logging, on your infrastructure, with great support for ASP.NET Core and Serilog. Version 4 adds integrated dashboards and alerts - check it out!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Hosting By

I was on a call with my co-worker Maria today and she commented on how nice my command prompt in Windows looked. I told it was "Solarized" and then our conference call fell apart as we collected all kinds of fun info about how you can get Solarized in your favorite apps on Windows.

Solarized is a sixteen color palette (eight monotones, eight accent colors) designed for use with terminal and gui applications. It's by Ethan Schoonover and it's spread all over the web. You can see screenshots and learn about it on GitHub.

Solarized for your Windows Command Prompt (cmd, powershell, bash)

By default when you right click and hit properties on a shortcut for a prompt like cmd, powershell, or bash, you'll get a dialog that looks like this.

You'll see there's 16 colors, usually 8 colors on the left, and then the "light/intense/bold" version of each color on the right. I usually used Intense Terminal Green on black before Solarized.

Those values (the defaults) are stored in the registry here HKEY_CURRENT_USER\Console

Those defaults are used for NEW shortcuts or consoles that start afresh, via Windows+R. This won't change existing shortcuts you may already have created. There's a few ways to fix this.

I've found the easiest manual way is to recreate the shortcuts. You can do this by just copy-pasting a shortcut and using the new one.

However, there is talk of programmatically updating .lnk (Start Menu link files) with PowerShell.

You'd just go to the location of each LNK file you want to change, then run Update-Link.ps1 YOURLINK.LNK "light|dark" and it'll load up the .lnk file using Windows APIs and save it with a new Color Table.

I've started that work here and I'll PR the main repo if I can solve one issue - I can't get it to switch to Solarized Light, just Dark. It might be something wrong on my side. Please take a look if you're a Win32/PowerShell internals type.

Here I went to where the Start Menu stores most of the LNK files. You can also search for an item in your start and right-click "Open File Location."pow

Here's before and after with my Developer Command Prompt for Visual Studio 2015.

NOTE: Once this is done, in cmd.exe you can also switch between light and dark with "color f6" or "color 01" which is nice for presentations. I'm not sure how to do this yet in PowerShell or Bash.

Here is the palette after:

For PowerShell there is also an extra-step you'll want to put into your Microsoft.PowerShell_profile.ps1 where you map things like Errors, Progress Bars, and Warnings internally in PowerShell. Be sure to read the instructions.

Solarized in Visual Studio and Visual Studio Code

As for Visual Studio and Visual Studio Code, they're far easier. You can just Ctrl-K then Ctrl-T in VSCode and pick Solarized.

For Visual Studio (all versions) you can head over to @leddt's GitHub and download settings files for Solarized that you can then import info VS from Tools | Import and Export Settings.

Sponsor: Big thanks to Raygun! Don't rely on your users to report the problems they experience. Automatically detect, diagnose and understand the root cause of errors, crashes and performance issues in your web and mobile apps. Learn more.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Hosting By