Scott Hanselman

Trying out new .NET Core Alpine Docker Images

November 22, '17 Comments [3] Posted in Docker | DotNetCore | Open Source
Sponsored By

Docker ContainersI blogged recently about optimizing .NET and ASP.NET Docker files sizes. .NET Core 2.0 has previously been built on a Debian image but today there is preview image with .NET Core 2.1 nightlies using Alpine. You can read about the announcement here about this new Alpine preview image. There's also a good rollup post on .NET and Docker.

They have added two new images:

  • 2.1-runtime-alpine
  • 2.1-runtime-deps-alpine

Alpine support is part of the .NET Core 2.1 release. .NET Core 2.1 images are currently provided at the microsoft/dotnet-nightly repo, including the new Alpine images. .NET Core 2.1 images will be promoted to the microsoft/dotnet repo when released in 2018.

NOTE: The -runtime-deps- image contains the dependancies needed for a .NET Core application, but NOT the .NET Core runtime itself. This is the image you'd use if your app was a self-contained application that included a copy of the .NET Core runtime. This is apps published with -r [runtimeid]. Most folks will use the -runtime- image that included the full .NET Core runtime. To be clear:

- The runtime image contains the .NET Core runtime and is intended to run Framework-Dependent Deployed applications - see sample

- The runtime-deps image contains just the native dependencies needed by .NET Core and is intended to run Self-Contained Deployed applications - see sample

It's best with .NET Core to use multi-stage build files, so you have one container that builds your app and one that contains the results of that build. That way you don't end up shipping an image with a bunch of SDKs and compilers you don't need.

NOTE: Read this to learn more about image versions in Dockerfiles so you can pick the right tag and digest for your needs. Ideally you'll pick a docker file that rolls forward to include the latest servicing patches.

Given this docker file, we build with the SDK image, then publish, and the result is about 219megs.

FROM microsoft/dotnet:2.0-sdk as builder  

RUN mkdir -p /root/src/app/dockertest
WORKDIR /root/src/app/dockertest

COPY dockertest.csproj .
RUN dotnet restore ./dockertest.csproj

COPY . .
RUN dotnet publish -c release -o published

FROM microsoft/dotnet:2.0.0-runtime

WORKDIR /root/
COPY --from=builder /root/src/app/dockertest/published .
ENV ASPNETCORE_URLS=http://+:5000
EXPOSE 5000/tcp
CMD ["dotnet", "./dockertest.dll"]

Then I'll save this as Dockerfile.debian and build like this:

> docker build . -t shanselman/dockertestdeb:0.1 -f dockerfile.debian

With a standard ASP.NET app this image ends up being 219 megs.

Now I'll just change one line, and use the 2.1 alpine runtime

FROM microsoft/dotnet-nightly:2.1-runtime-alpine

And build like this:

> docker build . -t shanselman/dockertestalp:0.1 -f dockerfile.alpine

and compare the two:

> docker images | find /i "dockertest"
shanselman/dockertestalp 0.1 3f2595a6833d 16 minutes ago 82.8MB
shanselman/dockertestdeb 0.1 0d62455c4944 30 minutes ago 219MB

Nice. About 83 megs now rather than 219 megs for a Hello World web app. Now the idea of a microservice is more feasible!

Please do head over to the GitHub issue here https://github.com/dotnet/dotnet-docker-nightly/issues/500 and offer your thoughts and results as you test these Alpine images. Also, are you interested in a "-debian-slim?" It would be halfway to Alpine but not as heavy as just -debian.

Lots of great stuff happening around .NET and Docker. Be sure to also check out Jeff Fritz's post on creating a minimal ASP.NET Core Windows Container to see how you can squish .(full) Framework applications running on Windows containers as well. For example, the Windows Nano Server images are just 93 megs compressed.


Sponsor: Get the latest JetBrains Rider preview for .NET Core 2.0 support, Value Tracking and Call Tracking, MSTest runner, new code inspections and refactorings, and the Parallel Stacks view in debugger.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Lightweight bundling, minifying, and compression, for CSS and JavaScript with ASP.NET Core and Smidge

November 8, '17 Comments [14] Posted in ASP.NET | DotNetCore | Open Source
Sponsored By

Yesterday I blogged about WebOptimizer, a minifier that Mads Kristensen wrote for ASP.NET Core. A few people mentioned that Shannon Deminick also had a great minifier for ASP.NET Core. Shannon has a number of great libraries on his GitHub https://github.com/Shazwazza including not just "Smidge" but also Examine, an indexing system, ClientDependency for managing all your client side assets, and Articulate, a blog engine built on Umbraco.

Often when there's more than one way to do things, but one of the ways is made by a Microsoft employee like Mads - even if it's in his spare time - it can feel like inside baseball or an unfair advantage. The same would apply if I made a node.js library but a node.js core committer also made a similar one. Many things can affect whether an open source library "pops," and it's not always merit. Sometimes it's locale/location, niceness of docs, marketing, word of mouth, website. Both Mads and Shannon and a dozen other people are all making great libraries and useful stuff. Sometimes people are aware of other projects and sometimes they aren't. At some point a community wants to "pick a winner" but even as I write this blog post, someone else we haven't met yet is likely making the next great bundler/minifier. And that's OK!

I'm going to take a look at Shannon Deminck's "Smidge" in this post. Smidge has been around as a runtime bundler since the beginning of ASP.NET Core even back when DNX was a thing, if you remember that. Shannon's been updating the library as ASP.NET Core has evolved, and it's under active development.

Smidge supports minification, combination, compression for JS/CSS files and features a fluent syntax for creating and configuring bundles

I'll start from "dotnet new mvc" and then:

C:\Users\scott\Desktop\smidgenweb>dotnet add package smidge
Writing C:\Users\scott\AppData\Local\Temp\tmp325B.tmp
info : Adding PackageReference for package 'smidge' into project 'C:\Users\scott\Desktop\smidgenweb\smidgenweb.csproj'.
log : Restoring packages for C:\Users\scott\Desktop\smidgenweb\smidgenweb.csproj...
...SNIP...
log : Installing Smidge 3.0.0.
info : Package 'smidge' is compatible with all the specified frameworks in project 'C:\Users\scott\Desktop\smidgenweb\smidgenweb.csproj'.
info : PackageReference for package 'smidge' version '3.0.0' added to file 'C:\Users\scott\Desktop\smidgenweb\smidgenweb.csproj'.

Then I'll update appSettings.json (where logging lives) and add Smidge's config:

{
"Logging": {
"IncludeScopes": false,
"LogLevel": {
"Default": "Warning"
}
},
"smidge": {
"dataFolder" : "App_Data/Smidge",
"version" : "1"
}
}

Let me squish my CSS, so I'll make a bundle:

app.UseSmidge(bundles =>
{
bundles.CreateCss("my-css", "~/css/site.css");
});

I refer to the bundle by name and the Smidge tag helper turns this:

<link rel="stylesheet" href="my-css" /> 

into this

<link href="/sb/my-css.css.v1" rel="stylesheet" />

Notice the generated filename with version embedded. That bundle could be one or more files, a whole folder, whatever you need.

Her eyou can see Kestral handling the request. Smidge jumps in there and does its thing, then the bundle is cached for the next request!

info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
Executing action method Smidge.Controllers.SmidgeController.Bundle (Smidge) with arguments (Smidge.Models.BundleRequestModel) - ModelState is Valid
dbug: Smidge.Controllers.SmidgeController[0]
Processing bundle 'my-css', debug? False ...
dbug: Smidge.FileProcessors.PreProcessManager[0]
Processing file '/css/site.css', type: Css, cacheFile: C:\Users\scott\Desktop\smidgenweb\App_Data\Smidge\Cache\SONOFHEXPOWER\1\bb8368ef.css, watching? False ...
dbug: Smidge.FileProcessors.PreProcessManager[0]
Processed file '/css/site.css' in 19ms
dbug: Smidge.Controllers.SmidgeController[0]
Processed bundle 'my-css' in 73ms
info: Microsoft.AspNetCore.Mvc.Internal.VirtualFileResultExecutor[1]
Executing FileResult, sending file

The minified results are cached wherever you want (remember I said App_Data):

Compressed JS and CSS

This is a SUPER simple example. You can use Smidge's fluent interface to affect how an individual bundle is created and behaves:

bundles.CreateJs("test-bundle-3", "~/Js/Bundle3")
.WithEnvironmentOptions(BundleEnvironmentOptions.Create()
.ForDebug(builder => builder
.EnableCompositeProcessing()
.EnableFileWatcher()
.SetCacheBusterType<AppDomainLifetimeCacheBuster>()
.CacheControlOptions(enableEtag: false, cacheControlMaxAge: 0))
.Build()
);

Smidge is unique in its Custom Pre-Processing Pipeline. Similar to ASP.NET Core itself, if there's anything you don't like or any behavior you want to change, you can.

I'm sure Shannon would appreciate help in Documentation and Open Issues, so go check out Smidge at https://github.com/Shazwazza/Smidge!


Sponsor: Check out JetBrains Rider: a new cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

WebOptimizer - a Bundler and Minifier for ASP.NET Core

November 8, '17 Comments [19] Posted in ASP.NET | DotNetCore | Open Source
Sponsored By

ASP.NET Core didn't have a runtime bundler like previous versions of ASP.NET. This was a bummer as I was a fan. Fortunately Mads Kristensen created one and put it on GitHub, called WebOptimizer.

WebOptimizer - ASP.NET Core middleware for bundling and minification of CSS and JavaScript files at runtime. With full server-side and client-side caching to ensure high performance.

I'll try it out on a default ASP.NET Core 2.0 app.

First, assuming I've installed http://dot.net I'll run

C:\Users\scott\Desktop> cd squishyweb

C:\Users\scott\Desktop\squishyweb> dotnet new mvc
The template "ASP.NET Core Web App (Model-View-Controller)" was created successfully.
This template contains technologies from parties other than Microsoft, see https://aka.ms/template-3pn for details.

SNIP

Restore succeeded.

Then I'll add a reference to the WebOptimizer package. Be sure to check the versioning and pick the one you want, or use the latest.

C:\Users\scott\Desktop\squishyweb> dotnet add package LigerShark.WebOptimizer.Core --version 1.0.178-beta 

Add the service in ConfigureServices and add it (I'll do it conditionally, only when in Production) in Configure. Notice I had to put it before UseStaticFiles() because I want it to get the first chance at those requests.

public void ConfigureServices(IServiceCollection services)
{
services.AddMvc();
services.AddWebOptimizer();
}

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
else
{
app.UseWebOptimizer();
app.UseExceptionHandler("/Home/Error");
}

app.UseStaticFiles();

app.UseMvc(routes =>
{
routes.MapRoute(
name: "default",
template: "{controller=Home}/{action=Index}/{id?}");
});
}

After running "dotnet run" I'll request site.css as an example and see it's automatically minimized:

CSS minification automatically

You can control the pipeline with globbing like this:

public void ConfigureServices(IServiceCollection services)
{
services.AddMvc();
services.AddWebOptimizer(pipeline =>
{
pipeline.MinifyJsFiles("js/a.js", "js/b.js", "js/c.js");
});
}

If I wanted to combine some files into an output "file" that'll be held/cached only in memory, I can do that also. To be clear, it'll never touch the disk, it's just a URL. Then I can just refer to it with a <link> within my Razor Page or main Layout.

services.AddWebOptimizer(pipeline =>
{
pipeline.AddCssBundle("/css/mybundle.css", "css/*.css");
});

WebOptimizer also supports automatic "cache busting" with a ?v= query string created by a TagHelper. It can even compile Scss (Sass) files into CSS. There's plugins for TypeScript, Less, and Markdown too!

WebOptimizer is open source and lives at https://github.com/ligershark/WebOptimizer. Go check it out, kick the tires, and see if it meets your needs! Maybe get involved and make a fix or help with docs! There are already some open issues you could start helping with.


Sponsor: Check out JetBrains Rider: a new cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Optimizing ASP.NET Core Docker Image sizes

October 31, '17 Comments [11] Posted in Docker | DotNetCore | Open Source
Sponsored By

ASP.NET Core on KubernetesThere is a great post from Steve Laster in 2016 about optimizing ASP.NET Docker Image sizes. Since then Docker has added multi-stage build files so you can do more in one Dockerfile...which feels like one step even though it's not. Containers are about easy and reliable deployment, and they're also about density. You want to use as little memory as possible, sure, but it also is nice to make them as small as possible so you're not spending time moving them around the network. The size of the image file can also affect startup time for the container. Plus it's just tidy.

I've been building a little 6 node Raspberry Pi (ARM) Kubenetes Cluster on my desk - like you do - this week, and I noticed that my image sizes were a little larger than I'd like. This is a bigger issue because it's a relatively low-powered system, but again, why carry around x unnecessary megabytes if you don't have to?

Alex Ellis has a great blog on building .NET Core apps for Raspberry Pi along with a YouTube video. In his video and blog he builds a "Console.WriteLine()" console app, which is great for OpenFaas (open source serverless platform) but I wanted to also have ASP.NET Core apps on my Raspberry Pi k8s cluster. He included this as a "challenge" in his blog, so challenge accepted! Thanks for all your help and support, Alex!

ASP.NET Core on Docker (on ARM)

First I make a basic ASP.NET Core app. I could do a Web API, but this time I'll do an MVC one with Razor Pages. To be clear, they are the same thing just with different starting points. I can always add pages or add JSON to either, later.

I start with "dotnet new mvc" (or dotnet new razor, etc). I'm going to be running this in Docker, managed by Kuberenetes, and while I can always change the WebHost in Program.cs to change how the Kestrel web server starts up like this:

WebHost.CreateDefaultBuilder(args)
.UseUrls(http://*:5000;http://localhost:5001;https://hostname:5002)

For Docker use cases it's easier to change the listening URL with an Environment Variable. Sure, it could be 80, but I like 5000. I'll set the ASPNETCORE_URLS environment variable to http://+:5000 when I make the Dockerfile.

Optimized MultiStage Dockerfile for ASP.NET

There's a number of "right" ways to do this, so you'll want to think about your scenarios. You'll see below that I'm using ARM (because Raspberry Pi) so if you see errors running your container like "qemu: Unsupported syscall: 345" then you're trying to run an ARM image on x86/x64. I'm going to be building an ARM container from Windows but I can't run it here. I have to push it to a container registry and then tell my Raspberry Pi cluster to pull it down and THEN it'll run, over there.

Here's what I have so far. NOTE there are some things commented out, so be conscious. This is/was a learning exercise for me. Don't you copy/paste unless you know what's up! And if there's a mistake, here's a GitHub Gist of my Dockerfile for you to change and improve.

It's important to understand that .NET Core has an SDK with build tools and development kits and compilers and stuff, and then it has a runtime. The runtime doesn't have the "make an app" stuff, it only has the "run an app stuff." There is not currently an SDK for ARM so that's a limitation that we are (somewhat elegantly) working around with the multistage build file. But, even if there WAS an SDK for ARM, we'd still want to use a Dockerfile like this because it's more efficient with space and makes a smaller image.

Let's break this down. There are two stages. The first FROM is the SDK image that builds the code. We're doing the build inside Docker - which is lovely, and  great reliable way to do builds.

PRO TIP: Docker is smart about making intermediate images and doing the least work, but it's useful if we (the authors) do the right thing as well to help it out.

For example, see where we COPY the .csproj over and then do a "dotnet restore"? Often you'll see folks do a "COPY . ." and then do a restore. That doesn't allow Docker to detect what's changed and you'll end up paying for the restore on EVERY BUILD.

By making this two steps - copy the project, restore, copy the code, this means your "dotnet restore" intermediate step will be cached by Docker and things will be WAY faster.

After you build, you'll do a publish. If you know the destination like I do (linux-arm) you can do a RID (runtime id) publish that is self-contained with -r linux-arm (or debian, or whatever) and you'll get a complete self-contained version of your app.

Otherwise, you can just publish your app's code and use a .NET Core runtime image to run it. Since I'm using a complete self-contained build for this image, it would be overkill to ALSO include the .NET runtime. If you look at the Docker hub for Microsoft/dotnet You'll see images called "deps" for "dependencies." Those are images that sit on top of debian that include the things .NET needs to run - but not .NET itself.

The stack of images looks generally like this (for example)

  • FROM debian:stretch
  • FROM microsoft/dotnet:2.0-runtime-deps
  • FROM microsoft/dotnet:2.0-runtime

So you have your base image, your dependencies, and your .NET runtime. The SDK image would include even more stuff since it needs to build code. Again, that's why we use that for the "as builder" image and then copy out the results of the compile and put them in another runtime image. You get the best of all worlds.

FROM microsoft/dotnet:2.0-sdk as builder  

RUN mkdir -p /root/src/app/aspnetcoreapp
WORKDIR /root/src/app/aspnetcoreapp

#copy just the project file over
# this prevents additional extraneous restores
# and allows us to re-use the intermediate layer
# This only happens again if we change the csproj.
# This means WAY faster builds!
COPY aspnetcoreapp.csproj .
#Because we have a custom nuget.config, copy it in
COPY nuget.config .
RUN dotnet restore ./aspnetcoreapp.csproj

COPY . .
RUN dotnet publish -c release -o published -r linux-arm

#Smaller - Best for apps with self-contained .NETs, as it doesn't include the runtime
# It has the *dependencies* to run .NET Apps. The .NET runtime image sits on this
FROM microsoft/dotnet:2.0.0-runtime-deps-stretch-arm32v7

#Bigger - Best for apps .NETs that aren't self-contained.
#FROM microsoft/dotnet:2.0.0-runtime-stretch-arm32v7

# These are the non-ARM images.
#FROM microsoft/dotnet:2.0.0-runtime-deps
#FROM microsoft/dotnet:2.0.0-runtime

WORKDIR /root/
COPY --from=builder /root/src/app/aspnetcoreapp/published .
ENV ASPNETCORE_URLS=http://+:5000
EXPOSE 5000/tcp
# This runs your app with the dotnet exe included with the runtime or SDK
#CMD ["dotnet", "./aspnetcoreapp.dll"]
# This runs your self-contained .NET Core app. You built with -r to get this
CMD ["./aspnetcoreapp"]

Notice also that I have a custom nuget.config, so if you do also you'll need to make sure that's available at build time for dotnet restore to pick up all packages.

I've included by commented out a bunch of the FROMs in the second stage. I'm using just the ARM one, but I wanted you to see the others.

Once we have the code we build copied into our runtime image, we set our environment variable so our all listens on port 5000 internally (remember that from above?) Then we run our app. Notice that you can run it with "dotnet foo.dll" if you have the runtime, but if you are like me and using a self-contained build, then you'll just run "foo."

To sum up:

  • Build with FROM microsoft/dotnet:2.0-sdk as builder
  • Copy the results out to a runtime
  • Use the right runtime FROM for you
    • Right CPU architecture?
    • Using the .NET Runtime (typical) or using a self-contained build (less so)
  • Listening on the right port (if a web app)?
  • Running your app successfully and correctly?
  • Do you have a .dockerignore? Super important for .NET Builds, as you don't' want to copy over /obj, /bin, etc, but you do want /published.
    obj/
    bin/
    !published/

Optimizing a little more

There are a few pre-release "Tree Trimming" tools that can look at your app and remove code and binaries that you are not calling. I included Microsoft.Packaging.Tools.Trimming as well to try it out and get even more unused code out of my final image by just adding a package to my project.

Step 8/14 : RUN dotnet publish -c release -o published -r linux-arm /p:LinkDuringPublish=true
---> Running in 39404479945f
Microsoft (R) Build Engine version 15.4.8.50001 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

Trimmed 152 out of 347 files for a savings of 20.54 MB
Final app size is 33.56 MB
aspnetcoreapp -> /root/src/app/aspnetcoreapp/bin/release/netcoreapp2.0/linux-arm/aspnetcoreapp.dll
Trimmed 152 out of 347 files for a savings of 20.54 MB
Final app size is 33.56 MB

If you run docker history on your final image you can see exactly where the size comes from. If/when Microsoft switches from a Debian base image to an Alpine one, this should get even smaller.

C:\Users\scott\Desktop\k8s for pi\aspnetcoreapp>docker history c60
IMAGE CREATED CREATED BY SIZE COMMENT
c6094ca46c3b 3 minutes ago /bin/sh -c #(nop) CMD ["dotnet" "./aspnet... 0B
b7dfcf137587 3 minutes ago /bin/sh -c #(nop) EXPOSE 5000/tcp 0B
a5ba51b91d9d 3 minutes ago /bin/sh -c #(nop) ENV ASPNETCORE_URLS=htt... 0B
8742269735bc 3 minutes ago /bin/sh -c #(nop) COPY dir:cc64bd3b9bacaeb... 56.5MB
28c008e38973 3 minutes ago /bin/sh -c #(nop) WORKDIR /root/ 0B
4bafd6e2811a 4 hours ago /bin/sh -c apt-get update && apt-get i... 45.4MB
<missing> 3 weeks ago /bin/sh -c #(nop) CMD ["bash"] 0B
<missing> 3 weeks ago /bin/sh -c #(nop) ADD file:8b7cf813a113aa2... 85.7MB

Here is the evolution of my Dockerfile as I made changes and the final result got smaller and smaller. Looks like 45 megs trimmed with a little work or about 20% smaller.

C:\Users\scott\Desktop\k8s for pi\aspnetcoreapp>docker images | find /i "aspnetcoreapp"
shanselman/aspnetcoreapp 0.5 c6094ca46c3b About a minute ago 188MB
shanselman/aspnetcoreapp 0.4 083bfbdc4e01 12 minutes ago 196MB
shanselman/aspnetcoreapp 0.3 fa053b4ee2b4 About an hour ago 199MB
shanselman/aspnetcoreapp 0.2 ba73f14e29aa 4 hours ago 207MB
shanselman/aspnetcoreapp 0.1 cac2f0e3826c 3 hours ago 233MB

Later I'll do a blog post where I put this standard ASP.NET Core web app into Kubernetes using this YAML description and scale it out on the Raspberry Pi. I'm learning a lot! Thanks to Alex Ellis and Glenn Condron and Jessie Frazelle for their time!


Sponsor: Create powerful Web applications to manage each step of a document’s life cycle with DocuVieware HTML5 Viewer and Document Management Kit. Check our demos to acquire, scan, edit, annotate 100+ formats, and customize your UI!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

How to Build a Kubernetes Cluster with ARM Raspberry Pi then run .NET Core on OpenFaas

October 28, '17 Comments [24] Posted in Open Source
Sponsored By

6 Raspberry Pi Kubernetes Cluster with Fabulous Batman on top

First, why would you do this? Why not. It's awesome. It's a learning experience. It's cheaper to get 6 pis than six "real computers." It's somewhat portable. While you can certainly quickly and easily build a Kubernetes Cluster in the cloud within your browser using a Cloud Shell, there's something more visceral about learning it this way, IMHO. Additionally, it's a non-trivial little bit of power you've got here. This is also a great little development cluster for experimenting. I'm very happy with the result.

By the end of this blog post you'll have not just Hello World but you'll have Cloud Native Distributed Containerized RESTful microservice based on ARMv7 w/ k8s Hello World! as a service. (original Tweet). ;)

Not familiar with why Kubernetes is cool? Check out Julia Evans' blog and read her K8s posts and you'll be convinced!

Hardware List (scroll down for Software)

Here's your shopping list. You may have a bunch of this stuff already. I had the Raspberry Pis and SD Cards already.

  • 6 - Raspberry Pi 3 - I picked 6, but you should have at least 3 or 4.
    • One Boss/Master and n workers. I did 6 because it's perfect for the power supply, perfect for the 8-port hub, AND it's a big but not unruly number.
  • 6 - Samsung 32Gb Micro SDHC cards - Don't be too cheap.
    • Faster SD cards are better.
  • 2x6 - 1ft flat Ethernet cables - Flat is the key here.
    • They are WAY more flexible. If you try to do this with regular 1ft cables you'll find them inflexible and frustrating. Get extras.
  • 1 - Anker PowerPort 6 Port USB Charging Hub - Regardless of this entire blog post, this product is amazing.
    • It's almost the same physical size as a Raspberry Pi, so it fits perfect at the bottom of your stack. It puts out 2.4a per port AND (wait for it) it includes SIX 1ft Micro USB cables...perfect for running 6 Raspberry Pis with a single power adapter.
  • 1 - 7 layer Raspberry Pi Clear Case Enclosure - I only used 6 of these, which is cool.
    • I love this case, and it looks fantastic.
  • 1 - Black Box USB-Powered 8-Port Switch - This is another amazing and AFAIK unique product.
    • An overarching goal for this little stack is that it be easy to move around and set up but also to power. We have power to spare, so I'd like to avoid a bunch of "wall warts" or power adapters. This is an 8 port switch that can be powered over a Raspberry Pi's USB. Because I'm given up to 2.4A to each micro USB, I just plugged this hub into one of the Pis and it worked no problem. It's also...wait for it...the size of a Pi. It also include magnets for mounting.
  • 1 - Some Small Router - This one is a little tricky and somewhat optional.
    • You can just put these Pis on your own Wifi and access them that way, but you need to think about how they get their IP address. Who doles out IPs via DHCP? Static Leases? Static IPs completely?
    • The root question is - How portable do you want this stack to be? I propose you give them their own address space and their own router that you then use to bridge to other places. Easiest way is with another router (you likely have one lying around, as I did. Could be any router...and remember hub/switch != router.
    • Here is a bad network diagram that makes the point, I think. The idea is that I should be able to go to a hotel or another place and just plug the little router into whatever external internet is available and the cluster will just work. Again, not needed unless portability matters to you as it does to me.
    • You could ALSO possibly get this to work with a Travel Router but then the external internet it consumed would be just Wifi and your other clients would get on your network subnet via Wifi as well. I wanted the relative predictability of wired.
    • What I WISH existed was a small router - similar to that little 8 port hub - that was powered off USB and had an internal and external Ethernet port. This ZyXEL Travel Router is very close...hm...
  • Optional - Pelican Case if you want portability. I'll see what airport security thinks. O_O
  • Optional - Tiny Keyboard and Mouse - Raspberry Pis can put out about 500mA per port for mice and keyboards. The number one problem I see with Pis is not giving them enough power and/or then having an external device take too much and then destabilize the system. This little keyboard is also a touchpad mouse and can be used to debug your Pi when you can't get remote access to it. You'll also want an HMDI cable occasionally.
  • You're Rich - If you have money to burn, get the 7" Touchscreen Display and a Case for it, just to show off htop in color on one of the Pis.

Dodgey Network Diagram

Network Diagram showing that the Pi Stack has its own Router

Disclaimer

OK, first things first, a few disclaimers.

The software in this space is moving fast. There's a non-zero chance that some of this software will have a new version out before I finish this blog post. In fact, when I was setting up Kubernetes, I created a few nodes, went to bed for 6 hours, came back and made a few more nodes and a new version had come out. Try to keep track, keep notes, and be aware of what works with what.

Kubernetes 1.8.1

Next, I'm just learning this stuff. I may get some of this wrong. While I've built (very) large distributed systems before, my experience with large orchestrators (primarily in banks) was with large proprietary ones in Java, C++, COM, and later in C#, .NET 1.x,2.0, and WCF. It's been really fascinating to see how Kubernetes thinks about these things and comparing it to how we thought about these things in the 90s and very early 2000s. A lot of best practices that were HUGE challenges many years ago are now being codified and soon, I hope, will "just work" for a new generation of developer. At least another full page of my resume is being marked [Obsolete] and I'm here for it. Things change and they are getting better.

Software

Get your Raspberry PIs and SD cards together. Also bookmark and subscribe to Alex Ellis' blog as you're going to find yourself there a lot. He's the author of OpenFaas, which I'll be using today and he's done a LOT of work making this experiment possible. So thank you Alex for being awesome! He has a great post on how Multi-stage Docker files make it possible to effectively use .NET Core on a Raspberry Pi while still building on your main machine. He and I spent a few late nights going around and around to make this easy.

Alex has put together a Gist we iterated on and I'll summarize here. You'll do these instructions n times for all machines.

You'll do special stuff for the ONE master/boss node and different stuff for the some number of worker nodes.

ADVANCED TIP! If you know what you're doing Linux-wise, you should save this excellent prep.sh shell script that Alex made, then SKIP to the node-specific instructions below. If you want to learn more, do it step by step.

ALL NODES

  • Burn Jessie to a SD Card
  • Creating an empty file called "ssh" before you put the card in the Raspberry Pi
  • SSH into the new Pi
    • I'm on Windows so I used WSL (Ubuntu) for Windows that lets me SSH and do run Linux natively.
    • ssh pi@raspberrypi
      • Login pi, password raspberry.
  • Change the Hostname

I ran

rasbpi-config

then immediately reboot with "sudo reboot"

  • Install Docker
curl -sSL get.docker.com | sh && \ sudo usermod pi -aG docker
  • Disable Swap. Important, you'll get errors in Kuberenetes otherwise

sudo dphys-swapfile swapoff && \ sudo dphys-swapfile uninstall && \ sudo update-rc.d dphys-swapfile remove
  • Go edit /boot/cmdline.txt with your favorite editor, or use
    sudo nano /boot/cmdline
    and add this at the very end. Don't press enter.
    cgroup_enable=cpuset cgroup_enable=memory
  • Install Kubernetes
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - && \ echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list && \ sudo apt-get update -q && \ sudo apt-get install -qy kubeadm 

MASTER/BOSS NODE

After ssh'ing into my main node, I used /ifconfig eth0 to figure out what the IP adresss was. Ideally you want this to be static (not changing) or at least a static lease. I logged into my router and set it as a static lease, so my main node ended up being 192.168.170.2, and .1 is the router itself.

Then I initialized this main node

sudo kubeadm init --apiserver-advertise-address=192.168.170.2

This took a WHILE. Like 10-15 min, so be patient.

Kubernetes uses this admin.conf for a ton of stuff, so you're going to want a copy in your $HOME folder so you can call "kubectl" easily later, copy it and take ownership.

mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config

When this is done, you'll get a nice print out with a ton of info and a token you have to save. Save it all. I took a screenshot.

The results of kubectl init

WORKER NODES

Ssh into your worker nodes and join them each to the main node. This line is the line you needed to have saved above when you did a kubectl init.

kubeadm join --token d758dc.059e9693bfa5 192.168.170.2:6443 --discovery-token-ca-cert-hash sha256:c66cb9deebfc58800a4afbedf0e70b93c086d02426f6175a716ee2f4d

Did it work?

While ssh'ed into the main node - or from any networked machine that has the admin.conf on it - try a few commands.

Here I'm trying "kubectl get nodes" and "kubectl get pods."

image

Note that I already have some stuff installed, so you'll want try "kubectl get pods --namespace kube-system" to see stuff running. If everything is "Running" then you can finish setting up networking. Kubernetes has fifty-eleven choices for networking and I'm not qualified to pick one. I tried Flannel and gave up and then tried Weave and it just worked. YMMV. Again, double check Alex's Gist if this changes.

kubectl apply -f https://git.io/weave-kube-1.6

At this point you should be ready to run some code!

Hello World...with Markdown

Back to Alex's gist, I'll try this "markdownrender" app. It will take some Markdown and return HTML.

Go get the function.yml from here and create the new app on your new cluster.

$ kubectl create -f function.yml
$ curl -4 http://localhost:31118 -d "# test"
<p><h1>test</h1></p>

This part can be tricky - it was for me. You need to understand what you're doing here. How do we know the ports? A few ways. First, it's listed as nodePort in the function.yml that represents the desired state of the application.

We can also run "kubectl get svc" and see the ports for various services.

pi@hanselboss1:~ $ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
alertmanager NodePort 10.103.43.130 <none> 9093:31113/TCP 1d
dotnet-ping ClusterIP 10.100.153.185 <none> 8080/TCP 1d
faas-netesd NodePort 10.103.9.25 <none> 8080:31111/TCP 2d
gateway NodePort 10.111.130.61 <none> 8080:31112/TCP 2d
http-ping ClusterIP 10.102.150.8 <none> 8080/TCP 1d
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 2d
markdownrender NodePort 10.104.121.82 <none> 8080:31118/TCP 1d
nodeinfo ClusterIP 10.106.2.233 <none> 8080/TCP 1d
prometheus NodePort 10.98.110.232 <none> 9090:31119/TCP 2d

See those ports that are outside:insider? You can get to markdownrender directly from 31118 on an internal IP like localhost, or the main/master IP. Those 10.x.x.x are all software networking, you can not worry about them. See?

pi@hanselboss1:~ $ curl -4 http://192.168.170.2:31118 -d "# test"
<h1>test</h1>

pi@hanselboss1:~ $ curl -4 http://10.104.121.82:31118 -d "# test"
curl: (7) Failed to connect to 10.104.121.82 port 31118: Network is unreachable

Can we access this cluster from another machine? My Windows laptop, perhaps?

Access your Raspberry Pi Kubernetes Cluster from your Windows Machine (or elsewhere)

I put KubeCtl on my local Windows machine put it in the PATH.

  • I copied the admin.conf over from my Raspberry Pi. You will likely use scp or WinSCP.
  • I made a little local batch file like this. I may end up with multiple clusters and I want it easy to switch between them.
    • SET KUBECONFIG="C:\users\scott\desktop\k8s for pi\admin.conf

Once you have Kubectl on another machine that isn't your Pi, try running "kubectl proxy" and see if you can hit your cluster like this. Remember you'll get weird "Connection refused" if kubectl thinks you're talking to a local cluster.

image

Here you can get to localhost:8001/api and move around, then you've successfully punched a hole over to your cluster (proxied) and you can treat localhost:8001 as your cluster. So "kubectl proxy" made that possible.

If you have WSL (Windows Subsystem for Linux) - and you should - then you could also do this and TUNNEL to the API. But I'm going to get cert errors and generally get frustrated. However, tunneling like this to other apps from Windows or elsewhere IS super useful. What about the Kubernetes Dashboard?

~ $ sudo ssh -L 8001:10.96.0.1:443 pi@192.168.170.2

I'm going to install the Kubernetes Dashboard like this:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/alternative/kubernetes-dashboard-arm.yaml

Pay close attention to that URL! There are several sites out there that may point to older URLs, non ARM dashboard, or use shortened URLs. Make sure you're applying the ARM dashboard. I looked here https://github.com/kubernetes/dashboard/tree/master/src/deploy.

Notice I'm using the "alternative" dashboard. That's for development and I'm saying I don't care at all about security when accessing it. Be aware.

I can see where my Dashboard is running, the port and the IP address.

pi@hanselboss1:~ $ kubectl get svc --namespace kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP 2d
kubernetes-dashboard ClusterIP 10.98.2.15 <none> 80/TCP 2d

NOW I can punch a hole with that nice ssh tunnel...

~ $ sudo ssh -L 8080:10.98.2.15:80 pi@192.168.170.2

I can access the Kubernetes Dashboard now from my Windows machine at http://localhost:8080 and hit Skip to login.

Kubernetes Dashboard

Do note the Namespace dropdown and think about what you're viewing. There's the kube-system stuff that manages the cluster

Adding OpenFaas and calling a serverless function

Let's go to the next level. We'll install OpenFaas - think Azure Functions or Amazon Lambda, except for your own Docker and Kubernetes cluster. To be clear, OpenFaas is an Application that we will run on Kubernetes, and it will make it easier to run other apps. Then we'll run other stuff on it...just some simple apps like Hello World in Python and .NET Core. OpenFaas is one of several open source "Serverless" solutions.

Do you need to use OpenFaas? No. But if your goal is to write a DoIt() function and put it on your little cluster easily and scale it out, it's pretty fabulous.

Remember my definition of Serverless...there ARE servers, you just don't think about them.

Serverless Computing is like this - Your code, a slider bar, and your credit card.

Let's go.

.NET Core on OpenFaas on Kubernetes on Raspberry Pi

I ssh'ed into my main/master cluster Pi and set up OpenFaas:

git clone https://github.com/alexellis/faas-netes && cd faas-netes 

kubectl apply -f faas.armhf.yml,rbac.yml,monitoring.armhf.yml

Once OpenFaas is installed on your cluster, here's Alex's great instructions on how to setup your first OpenFaas Python function, so give that a try first and test it. Once we've installed that Python function, we can also hit http://192.168.170.2:31112/ui/ (where that's your main Boss/Master's IP) and see it the OpenFaas UI.

OpenFaas and the "faas-netes" we setup above automates the build and deployment of our apps as Docker Images to Kuberetes. It makes the "Developer's Inner Loop" simpler. I'm going to make my .NET app, build, deploy, then change, build, deploy and I want it to "just work" on my cluster. And later, and I want it to scale.

OpenFaas Portal

I'm doing .NET Core, and since there is a runtime for .NET Core for Raspberry Pi (and ARM system) but no SDK, I need to do the build on my Windows machine and deploy from there.

Quick Aside: There are docker images for ARM/Raspberry PI for running .NET Core. However, you can't build .NET Core apps (yet?) directly ON the ARM machine. You have to build them on an x86/x64 machine and then get them over to the ARM machine. That can be SCP/FTPing them, or it can be making a docker container and then pushing that new docker image up to a container registry, then telling Kubernetes about that image. K8s (cool abbv) will then bring that ARM image down and run it. The technical trick that Alex and I noticed was of course that since you're building the Docker image on your x86/x64 machine, you can't RUN any stuff on it. You can build the image but you can't run stuff within it. It's an unfortunate limitation for now until there's a .NET Core SDK on ARM.

What's required on my development machine (not my Raspberry Pis?

Here's the gist we came up with, again thanks Alex! I'm going to do it from Windows.

I'll use the faas-cli to make a new function with charp. I'm calling mine dotnet-ping.

faas-cli new --lang csharp dotnet-ping

I'll edit the FunctionHandler.cs to add a little more. I'd like to know the machine name so I can see the scaling happen when it does.

using System;
using System.Text;

namespace Function
{
public class FunctionHandler
{
public void Handle(string input) {
Console.WriteLine("Hi your input was: "+ input + " on " + System.Environment.MachineName);
}
}
}

Check out the .yml file for your new OpenFaas function. Note the gateway IP should be your main Pi, and the port is 31112 which is OpenFaas.

I also changed the image to include "shanselman/" which is my Docker Hub. You could also use a local Container Registry if you like.

provider:
name: faas
gateway: http://192.168.170.2:31112

functions:
dotnet-ping:
lang: csharp
handler: ./dotnet-ping
image: shanselman/dotnet-ping

Head over to the ./template/csharp/Dockerfile and we're going to change it. Ordinarily it's fine if you are publishing from x64 to x64 but since we are doing a little dance, we are going to build and publish the .NET apps as linux-arm from our x64 machine, THEN push it, we'll use a multi stage docker file. Change the default Docker file to this:

FROM microsoft/dotnet:2.0-sdk as builder

ENV DOTNET_CLI_TELEMETRY_OPTOUT 1

# Optimize for Docker builder caching by adding projects first.

RUN mkdir -p /root/src/function
WORKDIR /root/src/function
COPY ./function/Function.csproj .

WORKDIR /root/src/
COPY ./root.csproj .
RUN dotnet restore ./root.csproj

COPY . .

RUN dotnet publish -c release -o published -r linux-arm

ADD https://github.com/openfaas/faas/releases/download/0.6.1/fwatchdog-armhf /usr/bin/fwatchdog
RUN chmod +x /usr/bin/fwatchdog

FROM microsoft/dotnet:2.0.0-runtime-stretch-arm32v7

WORKDIR /root/
COPY --from=builder /root/src/published .
COPY --from=builder /usr/bin/fwatchdog /

ENV fprocess="dotnet ./root.dll"
EXPOSE 8080
CMD ["/fwatchdog"]

Notice a few things. All the RUN commands are above the second FROM where we take the results of the first container and use its output to build the second ARM-based one. We can't RUN stuff because we aren't on ARM, right?

We use the Faas-Cli to build the app, build the docker container, AND publish the result to Kubernetes.

faas-cli build -f dotnet-ping.yml --parallel=1
faas-cli push -f dotnet-ping.yml
faas-cli deploy -f dotnet-ping.yml --gateway http://192.168.170.2:31112

And here is the dotnet-ping command running on the pi, as seen within the Kubernetes Dashboard.

I can then scale them out like this:

kubectl scale deploy/dotnet-ping --replicas=6

.NET on Raspberry Pi on Kubernetes 

And if I hit it multiple times - either via curl or via the dashboard, I see it's hitting different pods:

OpenFaas scales .NET apps

If I want to get super fancy, I can install Grafana - a dashboard manager by running locally in my machine on port 3000

docker run -p 3000:3000 -d grafana/grafana

Then I can add OpenFaas a datasource by pointing Grafana to http://192.168.170.2/31119 which is where the Prometheus metrics app is already running, then import the OpenFaas dashboard from the grafana.json file that is in the I cloned it from.

Grafana

Super cool. I'm going to keep using this little Raspberry Pi Kubernetes Cluster to learn as I get ready to do real K8s in Azure! Thanks to Alex Ellis for his kindness and patience and to Jessie Frazelle for making me love both Windows AND Linux!

* If you like this blog, please do use my Amazon links as they help pay for projects like this! They don't make me rich, but a few dollars here and there can pay for Raspberry Pis!

Sponsor: Check out JetBrains Rider: a new cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb
Page 1 of 46 in the Open Source category Next Page

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.