Scott Hanselman

How to parse string dates with a two digit year and split on the right century in C#

March 7, '19 Comments [8] Posted in DotNetCore | Learning .NET
Sponsored By

So you've been asked to parse some dates, except the years are two digit years. For example, dates like "12 Jun 30" are ambiguous...or are they?

If "12 Jun 30" is intended to express a birthday, given it's 2019 as the of writing of this post, we can assume it means 1930. But if the input is "12 Jun 18" is that last year, or is that a 101 year old person's birthday?

Enter the Calendar.TwoDigitYearMax property.

For example, if this property is set to 2029, the 100-year range is from 1930 to 2029. Therefore, a 2-digit value of 30 is interpreted as 1930, while a 2-digit value of 29 is interpreted as 2029.

The initial value for this property comes out of the DEPTHS of the region and languages portion of the Control Panel. Note way down there in "additional date, time, & regional settings" in the "more settings" and "date" tab, there's a setting that (currently) splits on 1950 and 2049.

Two Digit Year regional settings

If you're writing a server-side app that parses two digit dates you'll want to be conscious and explicit about what behavior you WANT so that you're not surprised.

Setting TwoDigitYearMax sets a 100 year RANGE that your two digit years will be interpreted to be within. You can also just change it on the current thread's current culture's calendar. It's up to you.

For example, this little app:

string dateString = "12 Jun 30"; //from user input
DateTime result;
CultureInfo culture = new CultureInfo("en-US");
DateTime.TryParse(dateString, culture, DateTimeStyles.None, out result);
Console.WriteLine(result.ToLongDateString());

culture.Calendar.TwoDigitYearMax = 2099;

DateTime.TryParse(dateString, culture, DateTimeStyles.None, out result);
Console.WriteLine(result.ToLongDateString());

gives this output:

Thursday, June 12, 1930
Wednesday, June 12, 2030

Note that I've changed TwoDigitYearMax from and moved it up to the 1999-2099 range so "30" is assumed to be 2030, within that 100 year range.

Hope this helps!


Sponsor: Stop wasting time trying to track down the cause of bugs. Sentry.io provides full stack error tracking that lets you monitor and fix problems in real time. If you can program it, we can make it far easier to fix any errors you encounter with it.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Converting an Excel Worksheet into a JSON document with C# and .NET Core and ExcelDataReader

March 6, '19 Comments [22] Posted in DotNetCore | Open Source
Sponsored By

Excel isn't a database, except when it isI've been working on a little idea where I'd have an app (maybe a mobile app with Xamarin or maybe a SPA, I haven't decided yet) for the easily accessing and searching across the 500+ videos from https://azure.microsoft.com/en-us/resources/videos/azure-friday/

HOWEVER. I don't have access to the database that hosts the metadata and while I'm trying to get at least read-only access to it (long story) the best I can do is a giant Excel spreadsheet dump that I was given that has all the video details.

This, of course, is sub-optimal, but regardless of how you feel about it, it's a database. Or, a data source at the very least! Additionally, since it was always going to end up as JSON in a cached in-memory database regardless, it doesn't matter much to me.

In real-world business scenarios, sometimes the authoritative source is an Excel sheet, sometimes it's a SQL database, and sometimes it's a flat file. Who knows?

What's most important (after clean data) is that the process one builds around that authoritative source is reliable and repeatable. For example, if I want to build a little app or one page website, yes, ideally I'd have a direct connection to the SQL back end. Other alternative sources could be a JSON file sitting on a simple storage endpoint accessible with a single HTTP GET. If the Excel sheet is on OneDrive/SharePoint/DropBox/whatever, I could have a small serverless function run when the files changes (or on a daily schedule) that would convert the Excel sheet into a JSON file and drop that file onto storage. Hopefully you get the idea. The goal here is clean, reliable pragmatism. I'll deal with the larger business process issue and/or system architecture and/or permissions issue later. For now the "interface" for my app is JSON.

So I need some JSON and I have this Excel sheet.

Turns out there's a lovely open source project and NuGet package called ExcelDataReader. There's been ways to get data out of Excel for decades. Literally decades. One of my first jobs was automating Microsoft Excel with Visual Basic 3.0 with COM Automation. I even blogged about getting data out of Excel into ASP.NET 16 years ago!

Today I'll use ExcelDataReader. It's really nice and it took less than an hour to get exactly what I wanted. I haven't gone and made it super clean and generic, refactored out a bunch of helper functions, so I'm interested in your thoughts. After I get this tight and reliable I'll drop it into an Azure Function and then focus on getting the JSON directly from the source.

A few gotchas that surprised me. I got a "System.NotSupportedException: No data is available for encoding 1252." Windows-1252 or CP-1252 (code page) is an old school text encoding (it's effectively ISO 8859-1). Turns out newer .NETs like .NET Core need the System.Text.Encoding.CodePages package as well as a call to System.Text.Encoding.RegisterProvider(System.Text.CodePagesEncodingProvider.Instance); to set it up for success. Also, that extra call to reader.Read at the start to skip over the Title row had me pause a moment.

using System;
using System.IO;
using ExcelDataReader;
using System.Text;
using Newtonsoft.Json;

namespace AzureFridayToJson
{
class Program
{
static void Main(string[] args)
{
Encoding.RegisterProvider(CodePagesEncodingProvider.Instance);

var inFilePath = args[0];
var outFilePath = args[1];

using (var inFile = File.Open(inFilePath, FileMode.Open, FileAccess.Read))
using (var outFile = File.CreateText(outFilePath))
{
using (var reader = ExcelReaderFactory.CreateReader(inFile, new ExcelReaderConfiguration()
{ FallbackEncoding = Encoding.GetEncoding(1252) }))
using (var writer = new JsonTextWriter(outFile))
{
writer.Formatting = Formatting.Indented; //I likes it tidy
writer.WriteStartArray();
reader.Read(); //SKIP FIRST ROW, it's TITLES.
do
{
while (reader.Read())
{
//peek ahead? Bail before we start anything so we don't get an empty object
var status = reader.GetString(0);
if (string.IsNullOrEmpty(status)) break;

writer.WriteStartObject();
writer.WritePropertyName("Status");
writer.WriteValue(status);

writer.WritePropertyName("Title");
writer.WriteValue(reader.GetString(1));

writer.WritePropertyName("Host");
writer.WriteValue(reader.GetString(6));

writer.WritePropertyName("Guest");
writer.WriteValue(reader.GetString(7));

writer.WritePropertyName("Episode");
writer.WriteValue(Convert.ToInt32(reader.GetDouble(2)));

writer.WritePropertyName("Live");
writer.WriteValue(reader.GetDateTime(5));

writer.WritePropertyName("Url");
writer.WriteValue(reader.GetString(11));

writer.WritePropertyName("EmbedUrl");
writer.WriteValue($"{reader.GetString(11)}player");
/*
<iframe src="https://channel9.msdn.com/Shows/Azure-Friday/Erich-Gamma-introduces-us-to-Visual-Studio-Online-integrated-with-the-Windows-Azure-Portal-Part-1/player" width="960" height="540" allowFullScreen frameBorder="0"></iframe>
*/

writer.WriteEndObject();
}
} while (reader.NextResult());
writer.WriteEndArray();
}
}
}
}
}

The first pass is on GitHub at https://github.com/shanselman/AzureFridayToJson and the resulting JSON looks like this:

[
{
"Status": "Live",
"Title": "Introduction to Azure Integration Service Environment for Logic Apps",
"Host": "Scott Hanselman",
"Guest": "Kevin Lam",
"Episode": 528,
"Live": "2019-02-26T00:00:00",
"Url": "https://azure.microsoft.com/en-us/resources/videos/azure-friday-introduction-to-azure-integration-service-environment-for-logic-apps",
"embedUrl": "https://azure.microsoft.com/en-us/resources/videos/azure-friday-introduction-to-azure-integration-service-environment-for-logic-appsplayer"
},
{
"Status": "Live",
"Title": "An overview of Azure Integration Services",
"Host": "Lara Rubbelke",
"Guest": "Matthew Farmer",
"Episode": 527,
"Live": "2019-02-22T00:00:00",
"Url": "https://azure.microsoft.com/en-us/resources/videos/azure-friday-an-overview-of-azure-integration-services",
"embedUrl": "https://azure.microsoft.com/en-us/resources/videos/azure-friday-an-overview-of-azure-integration-servicesplayer"
},
...SNIP...

Thoughts? There's a dozen ways to have done this. How would you do this? Dump it into a DataSet and serialize objects to JSON, make an array and do the same, automate Excel itself (please don't do this), and on and on.

Certainly this would be easier if I could get a CSV file or something from the business person, but the issue is that I'm regularly getting new drops of this same sheet with new records added. Getting the suit to Save As | CSV reliably and regularly isn't sustainable.


Sponsor: Stop wasting time trying to track down the cause of bugs. Sentry.io provides full stack error tracking that lets you monitor and fix problems in real time. If you can program it, we can make it far easier to fix any errors you encounter with it.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

EditorConfig code formatting from the command line with .NET Core's dotnet format global tool

March 1, '19 Comments [6] Posted in DotNetCore
Sponsored By

"EditorConfig helps maintain consistent coding styles for multiple developers working on the same project across various editors and IDEs." Rather than you having to keep your code in whatever format the team has agreed on, you can check in an .editorconfig file and your editor of choice will keep things in line.

If you're a .NET developer like myself, there's a ton of great .NET EditorConfig options you can set to ensure the team uses consistent Language Conventions, Naming Conventions, and Formatting Rules.

  • Language Conventions are rules pertaining to the C# or Visual Basic language, for example, var/explicit type, use expression-bodied member.
  • Formatting Rules are rules regarding the layout and structure of your code in order to make it easier to read, for example, Allman braces, spaces in control blocks.
  • Naming Conventions are rules respecting the way objects are named, for example, async methods must end in "Async".

If you're using Visual Studios 2010, 2012, 2013, or 2015, fear not. There's at least a basic EditorConfig free extension for you that enforces the basic rules. There is also an extension for Visual Studio Code to support EditorConfig files that takes just seconds to install.

ASIDE: If you are looking for a decent default for C#, take a look at the .editorconfig that the C# Roslyn compiler team uses. I don't know about you, but my brain exploded when I saw that they used spaces vs tabs.

But! What if you want this coding formatting goodness at the dotnet command line? You can use "dotnet format" as a global tool! It's one line to install, then it's available everywhere for all your .NET Core apps.

D:\github\hanselminutes-core> dotnet tool install -g dotnet-format
You can invoke the tool using the following command: dotnet-format
Tool 'dotnet-format' (version '3.0.2') was successfully installed.
D:\github\hanselminutes-core> dotnet format
Formatting code files in workspace 'D:\github\hanselminutes-core\hanselminutes-core.sln'.
Found project reference without a matching metadata reference: D:\github\hanselminutes-core\hanselminutes.core\hanselminutes-core.csproj
Formatting code files in project 'hanselminutes-core'.
Formatting code files in project 'hanselminutes.core.tests'.
Format complete.

You can see in the screenshot below where dotnet format used its scandalous defaults to move my end of line { to its own line! Well, if that's what the team wants! ;)

My code is automatically formatted by the dotnet format tool

Of course, dotnet format is all open source and up at https://github.com/dotnet/format. You can install the stable build OR a development build from myGet.


Sponsor: Manage GitHub Pull Requests right from the IDE with the latest JetBrains Rider. An integrated performance profiler on Windows comes to the rescue as well.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Hey Siri, what's my blood sugar? Learning to Code with Apple's iPhone Shortcuts

February 27, '19 Comments [5] Posted in Apple
Sponsored By

BA library of dozens of shortcuts on iOSear with me here. Apple Shortcuts (free on the App Store) is extraordinary and you shouldn't sleep on it. In fact, you should use it and explore it as it's amazing. I would go even further and say it could be a great place to learn to code!

Apple Shortcuts on iPhone is a lot like Microsoft Flow, except for your phone. Shortcuts let you string together Actions (ahem, functions) into multi-step tasks (ahem, functions that call functions). There's a rich and growing gallery of shortcuts that you can copy into your local (to your phone) library. You can then name them and invoke your Shortcuts with Siri.

Here's a few links to Shortcuts that (assuming you are reading this from your iPhone) you can add to your library with a click!

Once you have a shortcut you can invoke it as an item/icon on your springboard/home screen, you can have Siri run it with your voice, or invoke it via a "share sheet" that is available in all apps.

It would be reasonable to think this was a simple macro system with a few basic building blocks, but I don't think Apple's team gets enough credit. This is a complete development environment on your phone.

For example, here's a incredibly intricate and powerful Shortcut if one is pulled over by the police.

It pauses any music that may be playing, turns down your brightness and volume, turns on Do Not Disturb, and sends a message to the contact of your choosing letting them know you’re being pulled over and what your current location is. It then opens your front camera and starts a video recording so you have a video record of being pulled over.

Once you stop the recording it sends a copy of the video to a contact you specify, puts volume and brightness back to where they were, turns off Do Not Disturb, and gives you the option to send to iCloud Drive or Dropbox!

You could then record a Siri shortcut and just say "Hey Siri, I'm being pulled over" and all this happens automatically, hands free.

Take a look at the Laundry Timer app here. It's a very classic "take input and do a thing" program. You can build and extend workflows like this and the data from one flows through to the next one.

A multiple step shortcut with many actions that flow data into the next, organized in a pipeline

Note the Shortcut above. The "Adjust Date" action pops up a Date and is used as a Diff(erence) against the "Current Date" action, then used again in the Add New Reminder as an input to "Add New Reminder." These contextual variables flow through and are easily accessible in this genius UI. It really is near-perfect. Try it.

At this point you may be thinking, um, OK, that's cute, but where's the learn to code revolution here? It's not that open-ended of a system, what can I really do?

Like many connected cars, my car has a kind of REST API that its app uses to do things like heat up the climate system. Here I can literally POST (like Curl, but on your iPhone!) to an endpoint and pass in a FORM and parse the resulting JSON. Wow! Drink that in. You can write complex functions with iOS Shortcuts. Really.

calling a REST API with an iOS shortcut

Hang on. My body has a REST API. I use the open source Nightscout project to create a REST API on top of my Diabetes Continuous Glucose Meter then surface it in places like my lighted keyboard or even my Git Prompt.

How hard would it be to - right now as I make this blog post - write a method to have Siri retrieve my blood sugar and announce it to me when I say "Siri what's my blood sugar?" Let's see!

I make a URL object with my REST API that returns my sugar as JSON, it gets passed into Get Contents of URL. That makes a Dictionary from the Input, then gets the value of "sgv" (serum glucose value) and then the result of that is used to make a string with the Text action.

Preparing to make a shortcut

Now I have Siri SAY it. I can "debug" by running the Shortcut with the play button.

Building a shortcut

Then I can Add it to Siri and record my phrase. Here's me saying "what's my blood sugar" and she's telling me. Yes, I know. I had a cookie. I deserved it.

Running your shortcut

This is just the start. It could also tell me my trend lines, text someone if it's high, make a chart, I figure can do anything! I'm going to continue to explore Shortcuts but this little NightScout one can be downloaded to YOUR phone here. You'll only need to put in YOUR own URL for your Nightscout instance.


Sponsor: Manage GitHub Pull Requests right from the IDE with the latest JetBrains Rider. An integrated performance profiler on Windows comes to the rescue as well.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Learning about .NET Core futures by poking around at David Fowler's GitHub

February 22, '19 Comments [9] Posted in ASP.NET | DotNetCore | Open Source
Sponsored By

A picture of David silently judging my code, but with loveDavid Fowler is the ASP.NET Core Architect (and an amazing highly technical public speaker) and I've learned a lot from watching him code. However, what's the best way for YOU to learn from folks like David if you can't sit on their shoulder? Why, look at their GitHub!

Since .NET Core (and most of Microsoft) is not only open source but also developed in the open now on GitHub, we can actually watch folks in their day to day work as they commit code to projects like the C# compiler, .NET Core, and ASP.NET Core.

Even more interestingly, we can look at David's github here https://github.com/davidfowl and then under Repositories see what he's up to, filter by language and type, and explore! Sometimes I just explore the Pull Requests on projects like ASP.NET Core.

You can have Private repositories on GitHub, as I do, and as I'm sure David does. But GitHub is a social network for code and it's more fun and a better learning experience when we can see each others code and read it. Read with a critical eye, but without judgment as you may not have all the context that the author does. If you went to my GitHub, https://github.com/shanselman you might be disappointed but you also may be missing the big picture. Just consider that as you Follow people and explore their code.

David is an advanced .NET developer, while, for example, I am comparatively intermediate. So I realize that not all of David's code is FOR me. It's a scratchpad, it's not educational how-to workshops. However, I can get pick up cool idioms, interesting directions the tech may be going, and more importantly - prototypes and spikes. Spikes are folks testing out technical ideas. They may not be complete. In fact, they may never be complete. But some my be harbingers of things to come.

Here's a few things I learned today.

gRPC for .NET Core

For example, at https://github.com/davidfowl/grpc-dotnet I can see David has forked (copied) gRPC for dotnet and his game is working with the gRPC folks to make a fully supported version of gRPC for production workloads with .NET Core! Here are the stated goals:

  • We plan to implement a fully-managed version of gRPC for .NET that will be built on top of ASP.NET Core HTTP/2 server.
  • Good integration with the rest of ASP.NET Core ecosystem
  • High-performance (we plan to utilize some of the cutting edge performance features from ASP.NET Core and in .NET plaform itself)

That sounds cool! I can go learn that gRPC is a modern (google sponsored) Remote Procedure Call framework that can run anywhere. It's used by Netflix and Square and supports basically any languaige and any environment. Nice for this microservice world we are entering and hopefully has learned from the sins of DCOM and CORBA and RMI, because I was there and it sucked.

Nothing to see here but moving to a new JSON serializer

This Web.Framework sounds fun, and I'll be sure to take the description to heart.

says "Lame name, just a prototype, nothing to see here (move along)"

You can see David and James Newton-King kicking ideas around as you explore the commit log. However, the most interesting commit IMHO is when David moves this little spike from using JSON.NET (the ubiquitous 3rd party JSON serializer) to the new emerging official System.Text.Json. Here is the commit with unified differences.

It's a small change but it also makes me feel good about the API underneath this new JSON API that's coming. My takeway is that it's not as scary as I'd assumed. Looks like a Good Thing(tm).

A diff of code shows that just one line is changed to move JSON serializers

 

Cool!

Multi-Protocol ASP.NET Core

This looks interesting.

"The following sample shows how you can host a TCP server and HTTP server in the same ASP.NET Core application. Under the covers, it's the same server (Kestrel) running different protocols on different ports. The ConnectionHandler is a new primitive introduced in ASP.NET Core 2.1 to support non-HTTP protocols."

I didn't know you could do that! Looks like this sample hasn't changed much since it was conceived of in 2018, but then in the last month it's been updated twice and it appears to be part of a larger, slow-moving architectural issue called Bedrock that's moving forward.

I learned that Kestral (the ASP.NET Core web server) has a "ListenLocalhost" option on its options object!

WebHost.CreateDefaultBuilder(args)
.ConfigureServices(services =>
{
// This shows how a custom framework could plug in an experience without using Kestrel APIs directly
services.AddFramework(new IPEndPoint(IPAddress.Loopback, 8009));
})
.UseKestrel(options =>
{
// TCP 8007
options.ListenLocalhost(8007, builder =>
{
builder.UseConnectionHandler<MyEchoConnectionHandler>();
});

// HTTP 5000
options.ListenLocalhost(5000);

// HTTPS 5001
options.ListenLocalhost(5001, builder =>
{
builder.UseHttps();
});
})
.UseStartup<Startup>();

I can see here that TCP port 8007 is customer and uses a custom ConnectionHandler which I also didn't know existed! I can then look at the implementation of that handler and it's cool how clean the API is. You can get the result cleanly off the Transport buffer. You're doing low-level TCP but it doesn't feel low level.

using System.Threading.Tasks;
using Microsoft.AspNetCore.Connections;
using Microsoft.Extensions.Logging;

namespace KestrelTcpDemo
{
public class MyEchoConnectionHandler : ConnectionHandler
{
private readonly ILogger<MyEchoConnectionHandler> _logger;

public MyEchoConnectionHandler(ILogger<MyEchoConnectionHandler> logger)
{
_logger = logger;
}

public override async Task OnConnectedAsync(ConnectionContext connection)
{
_logger.LogInformation(connection.ConnectionId + " connected");

while (true)
{
var result = await connection.Transport.Input.ReadAsync();
var buffer = result.Buffer;

foreach (var segment in buffer)
{
await connection.Transport.Output.WriteAsync(segment);
}

if (result.IsCompleted)
{
break;
}

connection.Transport.Input.AdvanceTo(buffer.End);
}

_logger.LogInformation(connection.ConnectionId + " disconnected");
}
}
}

Pretty slick. This just echos what is sent to that port but not only has it educated me about a thing I didn't know about, it's something I can mentally file away until I need it!

All of these things I learned in just 30 minutes of exploring someone's public repository.

What kinds of code do you like to read and what have you learned from just poking around?


Sponsor: Get the latest JetBrains Rider for remote debugging via SSH, SQL injections, a new Search Everywhere popup, and improved Unity support.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.