Scott Hanselman

Carriage Returns and Line Feeds will ultimately bite you - Some Git Tips

June 5, '18 Comments [23] Posted in Linux | Win10
Sponsored By

Typewriter by Matunos used under Creative CommonsWhat's a Carriage and why is it Returning? Carriage Return Line Feed WHAT DOES IT ALL MEAN!?!

The paper on a typewriter rides horizontally on a carriage. The Carriage Return or CR was a non-printable control character that would reset the typewriter to the beginning of the line of text.

However, a Carriage Return moves the carriage back but doesn't advance the paper by one line. The carriage moves on the X axes...

And Line Feed or LF is the non-printable control character that turns the Platen (the main rubber cylinder) by one line.

Hence, Carriage Return and Line Feed. Two actions, and for years, two control characters.

Every operating system seems to encode an EOL (end of line) differently. Operating systems in the late 70s all used CR LF together literally because they were interfacing with typewriters/printers on the daily.

Windows uses CRLF because DOS used CRLF because CP/M used CRLF because history.

Mac OS used CR for years until OS X switched to LF.

Unix used just a single LF over CRLF and has since the beginning, likely because systems like Multics started using just LF around 1965. Saving a single byte EVERY LINE was a huge deal for both storage and transmission.

Fast-forward to 2018 and it's maybe time for Windows to also switch to just using LF as the EOL character for Text Files.

Why? For starters, Microsoft finally updated Notepad to handle text files that use LF.

BUT

Would such a change be possible? Likely not, it would break the world. Here's NewLine on .NET Core.

public static String NewLine {
    get {
        Contract.Ensures(Contract.Result() != null);
#if !PLATFORM_UNIX
        return "\r\n";
#else
        return "\n";
#endif // !PLATFORM_UNIX
    }
}

Regardless, if you regularly use Windows and WSL (Linux on Windows) and Linux together, you'll want to be conscious and aware of CRLF and LF.

I ran into an interesting situation recently. First, let's review what Git does

You can configure .gitattributes to tell Git how to to treat files, either individually or by extension.

When

git config --global core.autocrlf true

is set, git will automatically convert files quietly so that they are checked out in an OS-specific way. If you're on Linux and checkout, you'll get LF, if you're on Windows you'll get CRLF.

Viola on Twitter offers an important clarification:

"gitattributes controls line ending behaviour for a repo, git config (especially with --global) is a per user setting."

99% of the time system and the options available works great.

Except when you are sharing file systems between Linux and Windows. I use Windows 10 and Ubuntu (via WSL) and keep stuff in /mnt/c/github.

However, if I pull from Windows 10 I get CRLF and if I pull from Linux I can LF so then my shell scripts MAY OR MAY NOT WORK while in Ubuntu.

I've chosen to create a .gitattributes file that set both shell scripts and PowerShell scripts to LF. This way those scripts can be used and shared and RUN between systems.

*.sh eol=lf
*.ps1 eol=lf

You've got lots of choices. Again 99% of the time autocrlf is the right thing.

From the GitHub docs:

You'll notice that files are matched--*.c, *.sln, *.png--, separated by a space, then given a setting--text, text eol=crlf, binary. We'll go over some possible settings below.

  • text=auto
    • Git will handle the files in whatever way it thinks is best. This is a good default option.
  • text eol=crlf
    • Git will always convert line endings to CRLF on checkout. You should use this for files that must keep CRLF endings, even on OSX or Linux.
  • text eol=lf
    • Git will always convert line endings to LF on checkout. You should use this for files that must keep LF endings, even on Windows.
  • binary
    • Git will understand that the files specified are not text, and it should not try to change them. The binary setting is also an alias for -text -diff.

Again, the defaults are probably correct. BUT - if you're doing weird stuff, sharing files or file systems across operating systems then you should be aware.

Edward Thomson, a co-maintainer of libgit2, has this to say and points us to his blog post on Line Endings.

I would say this more strongly. Because `core.autocrlf` is configured in a scope that's per-user, but affects the way the whole repository works, `.gitattributes` should _always_ be used.

If you're having trouble, it's probably line endings. Edward's recommendation is that ALL projects check in a .gitattributes.

The key to dealing with line endings is to make sure your configuration is committed to the repository, using .gitattributes. For most people, this is as simple as creating a file named .gitattributes at the root of your repository that contains one line:
* text=auto

Hope this helps!

* Typewriter by Matunos used under Creative Commons


Sponsor: Check out JetBrains Rider: a cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

The year of Linux on the (Windows) Desktop - WSL Tips and Tricks

May 25, '18 Comments [13] Posted in Linux | Win10
Sponsored By

I've been doing a ton of work in bash/zsh/fish lately - Linuxing. In case you didn't know, Windows 10 can run Linux now. Sure, you can run Linux in a VM, but it's heavy and you need a decent machine. You can run a shell under Docker, but you'll need Hyper-V and Windows 10 Pro. You can even go to https://shell.azure.com and get a terminal anywhere - I do this on my Chromebook.

But mostly I run Linux natively on Windows 10. You can go. Just open PowerShell once, as Administrator and run this command and reboot:

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux

Then head over to the Windows Store and download Ubuntu, or Debian, or Kali, or whatever.

What's happening is you're running user-mode Linux without the Linux Kernel. The syscalls (system calls) that these un-modified Linuxes use are brokered over to Windows. Fork a Linux process? It a pico-process in Windows and shows up in the task manager.

Want to edit Windows files and edit them both in Windows and in Linux? Keep your files/code in /mnt/c/ and you can edit them with other OS. Don't use Windows to "reach into the Linux file system." There be dragons.

image

Once you've got a Linux installed (or many, as I do) you can manage then and use them in a number of ways.

Think this is stupid or foolish? Stop reading and keep running Linux and I wish you all the best. More power to you.

Want to know more? Want to look new and creative ways you can get the BEST of the Windows UI and Linux command line tools? Read on, friends.

wslconfig

WSL means "Windows Subsystem for Linux." Starting with the Windows 10 (version 1709 - that's 2017-09, the Fall Creators Update. Run "Winver" to see what you're running), you've got a command called "wslconfig." Try it out. It lists distros you have and controls which one starts when you type "bash."

Check out below that my default for "bash"  is Ubuntu 16.04, but I can run 18.04 manually if I like. See how I move from cmd into bash and exit out, then go back in, seamlessly. Again, no VM.

C:\>wslconfig /l /all
Windows Subsystem for Linux Distributions:
Ubuntu (Default)
Ubuntu-18.04
openSUSE-42
Debian
kali-rolling

C:\>wslconfig /l
Windows Subsystem for Linux Distributions:
Ubuntu (Default)
Ubuntu-18.04
openSUSE-42
Debian
kali-rolling

C:\>bash
128 → $ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.4 LTS
Release: 16.04
Codename: xenial
128 → $ exit
logout

C:\>ubuntu1804
scott@SONOFHEXPOWER:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04 LTS
Release: 18.04
Codename: bionic
scott@SONOFHEXPOWER:~$

You can also pipe things into Linux commands by piping to wsl or bash like this:

C:\Users\scott\Desktop>dir | wsl grep "poop"
05/18/2018 04:23 PM <DIR> poop

If you're in Windows, running cmd.exe or powershell.exe, it's best to move into Linux by running wsl or bash as it keeps the current directory.

C:\Users\scott\Desktop>bash
129 → $ pwd
/mnt/c/Users/scott/Desktop
129 → $ exit
logout

Cool! Wondering what that number is before my Prompt? That's my blood sugar. But that's another blog post.

wsl.conf

There's a file in /etc/wsl.conf that lets you control things like if your Linux of choice automounts your Windows drives. You can also control more advanced things like if Windows autogenerates a hosts file or processes /etc/fstab. It's up to you!

Distros

There's a half dozen distros available and more coming I'm told, but YOU can also make/package your own Linux distribution for WSL with packager/distro-launcher that's open sourced at GitHub.

Docker and WSL

Everyone wants to know if you can run Docker "natively" on WSL. No, that's a little too "Inception," and as mentioned, the Linux Kernel is not present. The unmodified elf binaries work fine but Windows does the work. BUT!

You can run Docker for Windows and click "Expose daemon on localhost:2375" and since Windows and WSL/Linux share the same port space, you CAN run the Docker client very happily on WSL.

After you've got Docker for Windows running in the background, install it in Ubuntu following the regular instructions. Then update your .bashrc to force your local docker client to talk to Docker for Windows:

echo "export DOCKER_HOST=tcp://0.0.0.0:2375" >> ~/.bashrc && source ~/.bashrc

There's lots of much longer and more details "Docker on WSL" tutorials, so if you'd like more technical detail, I'd encourage you to check them out! If you use a lot of Volume Mounts, I found Nick's write-up very useful.

Now when I run "docker images" or whatever from WSL I'm talking to Docker for Windows. Works great, exactly as you'd expect and you're sharing images and containers in both worlds.

128 → $ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
podcast test 1bd29d0223da 9 days ago 2.07GB
podcast latest e9dd366f0375 9 days ago 271MB
microsoft/dotnet-samples aspnetapp 80a65a6b6f95 11 days ago 258MB
microsoft/dotnet-samples dotnetapp b3d7f438bad3 2 weeks ago 180MB
microsoft/dotnet 2.1-sdk 1f63052e44c2 2 weeks ago 1.72GB
microsoft/dotnet 2.1-aspnetcore-runtime 083ca6a642ea 2 weeks ago 255MB
microsoft/dotnet 2.1-runtime 6d25f57ea9d6 2 weeks ago 180MB
microsoft/powershell latest 708fb186511e 2 weeks ago 318MB
microsoft/azure-cli latest 92bbcaff2f87 3 weeks ago 423MB
debian jessie 4eb8376dc2a3 4 weeks ago 127MB
microsoft/dotnet-samples latest 4070d1d1e7bb 5 weeks ago 219MB
docker4w/nsenter-dockerd latest cae870735e91 7 months ago 187kB
glennc/fancypants latest e1c29c74e891 20 months ago 291MB

Fabulous.

Coding and Editing Files

I need to hit this point again. Do not change Linux files using Windows apps and tools. However, you CAN share files and edit them with both Windows and Linux by keeping code on the Windows filesystem.

For example, my work is at c:\github so it's also at /mnt/c/github. I use Visual Studio code and edit my code there (or vim, from within WSL) and I run the code from Linux. I can even run bash/wsl from within Visual Studio Code using its integrated terminal. Just hit "Ctrl+P" in Visual Studio Code and type "Select Default Shell."

Select Default Shell in Visual Studio Code

On Windows 10 Insiders edition, Windows now has a UI called "Sets" that will give you Tabbed Command Prompts. Here I am installing Ruby on Rails in Ubuntu next to two other prompts - Cmd and PowerShell. This is all default Windows - no add-ons or extra programs for this experience.

Tabbed Command Prompts

I'm using Rails as an example here because Ruby/Rails support on Windows with native extensions has historically been a challenge. There's been a group of people heroically (and thanklessly) trying to get Ruby on Rails working well on Windows, but today there is no need. It runs great on Linux under Windows.

I can also run Windows apps or tools from Linux as long as I use their full name with extension (like code.exe) or set an alias.

Here I've made an alias "code" that runs code in the current directory, then I've got VS Code running editing my new Rails app.

Editing a Rails app on Linux on Windows 10 with VS Code

I can even mix and match Windows and Linux when piping. This will likely make Windows people happy and deeply offend Linux people. Or, if you're non-denominational like me, you'll dig it!

$ ipconfig.exe | grep IPv4 | cut -d: -f2
172.21.240.1
10.159.21.24

Again a reminder: Modifying files located not under /mnt/<x> with a Windows application in WSL is not supported. But edit stuff on /mnt/x with whatever and you're cool.

Sharing Sharing Sharing

If you have Windows 10 Build 17064 or newer (run ver from windows or "cmd.exe /c /ver" from Linux) and you can even share an environment variable!

131 → $ cmd.exe /c ver

Microsoft Windows [Version 10.0.17672.1000]

There's a special environment variable called "WSLENV" that is a colon-delimited list of environment variables that should be included when launching WSL processes from Win32 or Win32 processes from WSL. Basically you give it a list of variables you want to roam/share. This will make it easy for things like cross-platform dual builds. You can even add a /p flag and it'll automatically translate paths between c:\windows style and /mnt/c/windows style.

Check out the example at the WSL Blog about how to share a GOPATH and use VSCode in Windows and run Go in both places.

You can also use a special built-in command line called "wslpath" to translate path names between Windows and WSL. This is useful if you're sharing bash scripts, doing cross-platform scripts (I have PowerShell Core scripts that run in both places) or just need to programmatically switch path types.

131 → $ wslpath "d:\github\hanselminutes-core"
/mnt/d/github/hanselminutes-core
131 → $ wslpath "c:\Users\scott\Desktop"
/mnt/c/Users/scott/Desktop

There is no man page for wslpath yet, but copied from this GitHub issue, here's the gist:

wslpath usage:
-a force result to absolute path format
-u translate from a Windows path to a WSL path (default)
-w translate from a WSL path to a Windows path
-m translate from a WSL path to a Windows path, with ‘/’ instead of ‘\\’

One final note, once you've installed a Linux distro from the Windows Store, it's on you to keep it up to date. The Windows Store won't run "apt upgrade" or ever touch your Linuxes once they have been installed. Additionally, you can have Ubuntu 1604 and 1804 installed side-by-side and it won't hurt anything.

Related Links

Are you using WSL?


Sponsor: Check out JetBrains Rider: a cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Installing PowerShell Core on a Raspberry Pi (powered by .NET Core)

May 18, '18 Comments [3] Posted in Linux | Open Source | PowerShell
Sponsored By

PowerShell Core on a Raspberry Pi!Earlier this week I set up .NET Core and Docker on a Raspberry Pi and found that I could run my podcast website quite easily on a Pi. Check that post out as there's a lot going on. I can test within a Linux Container and output the test results to the host and then open them in VS. I also explored a reasonably complex Dockerfile that is both multiarch and multistage. I can reliably build and test my website either inside a container or on the bare metal of Windows or Linux. Very fun.

As primarily a Windows developer I have lots of batch/cmd files like "test.bat" or "dockerbuild.bat." They start as little throwaway bits of automation but as the project grows inevitably more complex.

I'm not interested in "selling" anyone PowerShell. If you like bash, use bash, it's lovely, as are shell scripts. PowerShell is object-oriented in its pipeline, moving lists of real objects as standard output. They are different and most importantly, they can live together. Just like you might call Python scripts from bash, you can call PowerShell scripts from bash, or vice versa. Another tool in our toolkits.

PS /home/pi> Get-Process | Where-Object WorkingSet -gt 10MB

NPM(K) PM(M) WS(M) CPU(s) Id SI ProcessName
------ ----- ----- ------ -- -- -----------
0 0.00 10.92 890.87 917 917 docker-containe
0 0.00 35.64 1,140.29 449 449 dockerd
0 0.00 10.36 0.88 1272 037 light-locker
0 0.00 20.46 608.04 1245 037 lxpanel
0 0.00 69.06 32.30 3777 749 pwsh
0 0.00 31.60 107.74 647 647 Xorg
0 0.00 10.60 0.77 1279 037 zenity
0 0.00 10.52 0.77 1280 037 zenity

Bash and shell scripts are SUPER powerful. It's a whole world. But it is text based (or json for some newer things) so you're often thinking about text more.

pi@raspberrypidotnet:~ $ ps aux | sort -rn -k 5,6 | head -n6
root 449 0.5 3.8 956240 36500 ? Ssl May17 19:00 /usr/bin/dockerd -H fd://
root 917 0.4 1.1 910492 11180 ? Ssl May17 14:51 docker-containerd --config /var/run/docker/containerd/containerd.toml
root 647 0.0 3.4 155608 32360 tty7 Ssl+ May17 1:47 /usr/lib/xorg/Xorg :0 -seat seat0 -auth /var/run/lightdm/root/:0 -nolisten tcp vt7 -novtswitch
pi 1245 0.2 2.2 153132 20952 ? Sl May17 10:08 lxpanel --profile LXDE-pi
pi 1272 0.0 1.1 145928 10612 ? Sl May17 0:00 light-locker
pi 1279 0.0 1.1 145020 10856 ? Sl May17 0:00 zenity --warning --no-wrap --text

You can take it as far as you like. For some it's intuitive power, for others, it's baroque.

pi@raspberrypidotnet:~ $ ps -eo size,pid,user,command --sort -size | awk '{ hr=$1/1024 ; printf("%13.2f Mb ",hr) } { for ( x=4 ; x<=NF ; x++ ) { printf("%s ",$x) } print "" }'
0.00 Mb COMMAND
161.14 Mb /usr/bin/dockerd -H fd://
124.20 Mb docker-containerd --config /var/run/docker/containerd/containerd.toml
78.23 Mb lxpanel --profile LXDE-pi
66.31 Mb /usr/lib/xorg/Xorg :0 -seat seat0 -auth /var/run/lightdm/root/:0 -nolisten tcp vt7 -novtswitch
61.66 Mb light-locker

Point is, there's choice. Here's a nice article about PowerShell from the perspective of a Linux user. Can I install PowerShell on my Raspberry Pi (or any Linux machine) and use the same scripts in both places? YES.

For many years PowerShell was a Windows-only thing that was part of the closed Windows ecosystem. In fact, here's video of me nearly 12 years ago (I was working in banking) talking to Jeffrey Snover about PowerShell. Today, PowerShell is open source up at https://github.com/PowerShell with lots of docs and scripts, also open source. PowerShell is supported on Windows, Mac, and a half-dozen Linuxes. Sound familiar? That's because it's powered (ahem) by open source cross platform .NET Core. You can get PowerShell Core 6.0 here on any platform.

Don't want to install it? Start it up in Docker in seconds with

docker run -it microsoft/powershell

Sweet. How about Raspbian on my ARMv7 based Raspberry Pi? I was running Raspbian Jessie and PowerShell is supported on Raspbian Stretch (newer) so I upgraded from Jesse to Stretch (and tidied up and did the firmware while I'm at it) with:

$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get dist-upgrade
$ sudo sed -i 's/jessie/stretch/g' /etc/apt/sources.list
$ sudo sed -i 's/jessie/stretch/g' /etc/apt/sources.list.d/raspi.list
$ sudo apt-get update && sudo apt-get upgrade -y
$ sudo apt-get dist-upgrade -y
$ sudo rpi-update

Cool. Now I'm on Raspbian Stretch on my Raspberry Pi 3. Let's install PowerShell! These are just the most basic Getting Started instructions. Check out GitHub for advanced and detailed info if you have issues with prerequisites or paths.

NOTE: Here I'm getting PowerShell Core 6.0.2. Be sure to check the releases page for newer releases if you're reading this in the future. I've also used 6.1.0 (in preview) with success. The next 6.1 preview will upgrade to .NET Core 2.1. If you're just evaluating, get the latest preview as it'll have the most recent bug fixes.

$ sudo apt-get install libunwind8
$ wget https://github.com/PowerShell/PowerShell/releases/download/v6.0.2/powershell-6.0.2-linux-arm32.tar.gz
$ mkdir ~/powershell
$ tar -xvf ./powershell-6.0.2-linux-arm32.tar.gz -C ~/powershell
$ sudo ln -s ~/powershell/pwsh /usr/bin/pwsh
$ sudo ln -s ~/powershell/pwsh /usr/local/bin/powershell
$ powershell

Lovely.

GOTCHA: Because I upgraded from Jessie to Stretch, I ran into a bug where libssl1.0.0 is getting loaded over libssl1.0.2. This is a complex native issue with interaction between PowerShell and .NET Core 2.0 that's being fixed. Only upgraded machines like mind will it it, but it's easily fixed with sudo apt-get remove libssl1.0.0

Now this means my PowerShell build scripts can work on both Windows and Linux. This is a deeply trivial example (just one line) but note the "shebang" at the top that lets Linux know what a *.ps1 file is for. That means I can keep using bash/zsh/fish on Raspbian, but still "build.ps1" or "test.ps1" on any platform.

#!/usr/local/bin/powershell
dotnet watch --project .\hanselminutes.core.tests test /p:CollectCoverage=true /p:CoverletOutputFormat=lcov /p:CoverletOutput=./lcov

Here's a few totally random but lovely PowerShell examples:

PS /home/pi> Get-Date | Select-Object -Property * | ConvertTo-Json
{
"DisplayHint": 2,
"DateTime": "Sunday, May 20, 2018 5:55:35 AM",
"Date": "2018-05-20T00:00:00+00:00",
"Day": 20,
"DayOfWeek": 0,
"DayOfYear": 140,
"Hour": 5,
"Kind": 2,
"Millisecond": 502,
"Minute": 55,
"Month": 5,
"Second": 35,
"Ticks": 636623925355021162,
"TimeOfDay": {
"Ticks": 213355021162,
"Days": 0,
"Hours": 5,
"Milliseconds": 502,
"Minutes": 55,
"Seconds": 35,
"TotalDays": 0.24693868190046295,
"TotalHours": 5.9265283656111105,
"TotalMilliseconds": 21335502.1162,
"TotalMinutes": 355.59170193666665,
"TotalSeconds": 21335.502116199998
},
"Year": 2018
}

You can take PowerShell objects to and from Objects, Hashtables, JSON, etc.

PS /home/pi> $hash | ConvertTo-Json
{
"Shape": "Square",
"Color": "Blue",
"Number": 1
}
PS /home/pi> $hash = @{ Number = 1; Shape = "Square"; Color = "Blue"}
PS /home/pi> $hash

Name Value
---- -----
Shape Square
Color Blue
Number 1


PS /home/pi> $hash | ConvertTo-Json
{
"Shape": "Square",
"Color": "Blue",
"Number": 1
}

Here's a nice one from MCPMag:

PS /home/pi> $URI = "https://query.yahooapis.com/v1/public/yql?q=select  * from weather.forecast where woeid in (select woeid from geo.places(1) where  text='{0}, {1}')&format=json&env=store://datatables.org/alltableswithkeys"  -f 'Omaha','NE'
PS /home/pi> $Data = Invoke-RestMethod -Uri $URI
PS /home/pi> $Data.query.results.channel.item.forecast|Format-Table

code date day high low text
---- ---- --- ---- --- ----
39 20 May 2018 Sun 62 56 Scattered Showers
30 21 May 2018 Mon 78 53 Partly Cloudy
30 22 May 2018 Tue 88 61 Partly Cloudy
4 23 May 2018 Wed 89 67 Thunderstorms
4 24 May 2018 Thu 91 68 Thunderstorms
4 25 May 2018 Fri 92 69 Thunderstorms
34 26 May 2018 Sat 89 68 Mostly Sunny
34 27 May 2018 Sun 85 65 Mostly Sunny
30 28 May 2018 Mon 85 63 Partly Cloudy
47 29 May 2018 Tue 82 63 Scattered Thunderstorms

Or a one-liner if you want to be obnoxious.

PS /home/pi> (Invoke-RestMethod -Uri  "https://query.yahooapis.com/v1/public/yql?q=select  * from weather.forecast where woeid in (select woeid from geo.places(1) where  text='Omaha, NE')&format=json&env=store://datatables.org/alltableswithkeys").query.results.channel.item.forecast|Format-Table

Example: This won't work on Linux as it's using Windows specific AIPs, but if you've got PowerShell on your Windows machine, try out this one-liner for a cool demo:

iex (New-Object Net.WebClient).DownloadString("http://bit.ly/e0Mw9w")

Thoughts?


Sponsor: Check out JetBrains Rider: a cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Cross-platform GUIs with open source .NET using Eto.Forms

March 16, '18 Comments [7] Posted in Linux | Open Source
Sponsored By

Amazing Cross Platform ANSI art editorThis is one of those "Did you know you could do THAT?" Many folks have figured out that C#/F#/.NET is cross-platform and open0source and runs on basically any operating system. People are using it to create micro services, web sites, and webAPI's all over. Not to mention iPhone/Android apps with Xamarin and video games with Unity and MonoGame.

But what about cross platform UIs?

While not officially supported by Microsoft - you can do some awesome stuff...as is how Open Source is supposed to work! Remember that there's a family of .NET Runtimes now, there's the .NET Framework on Windows, there's xplat .NET Core, and there's xplat Mono.

Eto.Forms has been in development since 2012 and is a cross-platform framework for creating GUI (Graphical User Interface, natch) applications with .NET that run across multiple platforms using their native toolkit. Not like Java in the 90s with custom painted buttons on canvas.

It's being used for real stuff! In fact, PabloDraw is an Ansi/Ascii text editor that you didn't know you needed in your life. But you do. It runs on Windows, Mac, and Linux and was written using Eto.Forms but has a native UI on each platform. Be sure to check out Curtis Wensley's Twitter account for some cool examples of what PabloDraw and Eto.Forms can do!

  • OS X: MonoMac or Xamarin.Mac (and also iOS via Xamarin)
  • Linux: GTK# 2 or 3
  • Windows: Windows Forms (using GDI or Direct2D) or WPF

Here's an example Hello World. Note that it's not just Code First, you can also use Xaml, or even Json (.jeto) to layout your forms!

using Eto.Forms;
using Eto.Drawing;

public class MyForm : Form
{
public MyForm ()
{
Title = "My Cross-Platform App";
ClientSize = new Size(200, 200);
Content = new Label { Text = "Hello World!" };
}

[STAThread]
static void Main()
{
new Application().Run(new MyForm());
}
}

Or I can just File | New Project with their Visual Studio Extension. You should definitely give it a try.

image

Even on the same platform (Windows in the below example) amazingly Eto.Forms can use whatever Native Controls you prefer. Here's a great example zip that has precompiled test apps.

WinForms, WPF, and Direct2D apps

Once you've installed a new version of Mono on Ubuntu, you can run the same sample as Gtk3, as I'm doing here in a VM. AMAZING.

image

Here's some example applications that are in the wild, using Eto.Forms:

There's so much cool stuff happening in open source .NET right now, and Eto.Forms is actively looking for help. Go check out their excellent Wiki, read the Tutorials, and maybe get involved!


Sponsor: Get the latest JetBrains Rider for debugging third-party .NET code, Smart Step Into, more debugger improvements, C# Interactive, new project wizard, and formatting code in columns.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Running ASP.NET Core on GoDaddy's cheapest shared Linux Hosting - Don't Try This At Home

March 1, '18 Comments [18] Posted in DotNetCore | Linux | Open Source
Sponsored By

First, a disclaimer. Don't do this. I did this to test a theory and to prove a point. ASP.NET Core and the .NET Core that it runs on are open source and run pretty much anywhere. I wanted to see if I could run an ASP.NET Core site on GoDaddy's cheapest hosting ($3, although it scales to $8) that basically supports only PHP. It's not a full Linux VM. It's locked-down and limited. You don't have root. You are missing most tools you'd expect you'd have.

BUT.

I wanted to see if I could get ASP.NET Core running on it anyway. Maybe if I do, they (and other inexpensive hosts) will talk to the .NET team, learn that ASP.NET Core is open source and could easily run on their existing infrastructure.

AGAIN: Don't do this. It's hacky. It's silly. But it's hella cool. IMHO. Also, big thanks to Tomas Weinfurt for his help!

First, I went to GoDaddy and signed up for their cheap hosting. Again, not a VM, but their shared one. I also registered supercheapaspnetsite.com as well. They use a cPanel-based web management system that doesn't really let you do anything. You can turn on SSH, do some PHP stuff, and generally poke around, but it's not exactly low-level.

First I ssh (shoosh!) in and see what I'm working with. I'm shooshing with Ubuntu on Windows 10 feature, that every developer should turn on. It's makes it really easy to work with Linux hosts if you're starting from Linux on Windows 10.

secretname@theirvmname [/proc]$ cat version
Linux version 2.6.32-773.26.1.lve1.4.46.el6.x86_64 (mockbuild@build.cloudlinux.com) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-18) (GCC) ) #1 SMP Tue Dec 5 18:55:41 EST 2017
secretname@theirvmname [/proc]$

OK, looks like Red Hat, so CentOS 6 should be compatible.

I'm going to use .NET Core 2.1 (which is in preview now!) and get the SDK at https://www.microsoft.com/net/download/all and install it on my Windows machine where I will develop and build the app. I don't NEED to use Windows to do this, but it's the laptop I have and it's also nice to know I can build on Windows but target CentOS/RHEL6.

Next I'll make a new ASP.NET site with

dotnet new razor

and then I'll publish a self-contained version like this:

dotnet publish -r rhel.6-x64

And those files will end up in a folder like \supercheapaspnetsite\bin\Debug\netcoreapp2.1\rhel.6-x64\publish\

NOTE: You may need to add the NuGet feed for the dailies for this .NET Core preview in order to get the RHEL6 runtime downloaded during this local publish.

Then I used WinSCP (or whatever FTP/SCP client you like, rsync, etc) to get the files over to the ~/www folder on your GoDaddy shared site. Then I

chmod +x ./supercheapasnetsite

to make it executable. Now, from my ssh session at GoDaddy, let's try to run my app!

secretname@theirvmname [~/www]$ ./supercheapaspnetsite
Failed to load hb, error: libunwind.so.8: cannot open shared object file: No such file or directory
Failed to bind to CoreCLR at '/home/secretname/public_html/libcoreclr.so'

Of course it couldn't be that easy, right? .NET Core wants the unwind library (shared object) and it doesn't exist on this locked down system.

AND I don't have yum/apt/rpm or a way to install it right?

I could go looking for tar.gz file somewhere like this http://download.savannah.nongnu.org/releases/libunwind/ but I need to think about versions and make sure things line up. Given that I'm targeting CentOS6, I should start here https://centos.pkgs.org/6/epel-x86_64/libunwind-1.1-3.el6.x86_64.rpm.html and download libunwind-1.1-3.el6.x86_64.rpm.

I need to crack open that rpm file and get the library. RPM packages are just headers on top of a CPIO archive, so I can apt-get install rpm2cpio from my local Ubuntu instances (on Windows 10). Then from /mnt/c/users/scott/Downloads (where I downloaded the file) I will extract it.

rpm2cpio ./libunwind-1.1-3.el6.x86_64.rpm | cpio -idmv

There they are.

image

This part is cool. Even though I have these files, I don't have root or any way to "install" them. However I could either export/use the LD_LIBRARY_PATH environment variable to control how libraries get loaded OR I could put these files in $ORIGIN/netcoredeps. You can read more about Self Contained Linux Applications on .NET Core here.

The main executable of published .NET Core applications (which is the .NET Core host) has an RPATH property set to $ORIGIN/netcoredeps. That means that when Linux shared library loader is looking for shared libraries, it looks to this location before looking to default shared library locations. It is worth noting that the paths specified by the LD_LIBRARY_PATHenvironment variable or libraries specified by the LD_PRELOAD environment variable are still used before the RPATH property. So, in order to use local copies of the third-party libraries, developers need to create a directory named netcoredeps next to the main application executable and copy all the necessary dependencies into it.

At this point I've added a "netcoredeps" folder to my public folder, and then copied it (scp) over to GoDaddy. Let's run it again.

secretname@theirvmname [~/www]$ ./supercheapaspnetsite
FailFast: Couldn't find a valid ICU package installed on the system. Set the configuration flag System.Globalization.Invariant to true if you want to run with no globalization support.

   at System.Environment.FailFast(System.String)
   at System.Globalization.GlobalizationMode.GetGlobalizationInvariantMode()
   at System.Globalization.GlobalizationMode..cctor()
   at System.Globalization.CultureData.CreateCultureWithInvariantData()
   at System.Globalization.CultureData.get_Invariant()
   at System.Globalization.CultureInfo..cctor()
   at System.StringComparer..cctor()
   at System.AppDomain.InitializeCompatibilityFlags()
   at System.AppDomain.Setup(System.Object)
Aborted

Ok, now it's complaining about ICU packages. These are for globalization. That is also mentioned in the self-contained-linux apps docs and there's a precompiled binary I could download. But there's options.

If your app doesn't explicitly opt out of using globalization, you also need to add libicuuc.so.{version}, libicui18n.so.{version}, and libicudata.so.{version}

I like "opt-out" so I don't have to go dig these ups (although I could) so I can either set the CORECLR_GLOBAL_INVARIANT env var to 1, or I can add System.Globalization.Invariant = true to supercheapaspnetsite.runtimeconfig.json, which I'll do with just to be obnoxious. ;)

When I run it again I get another complained about libuv. Yet another shared library that isn't installed on this instance. I could  go get it and put it in netcoredeps OR since I'm using the .NET Core 2.1, I could try something new. There were some improvements made in .NET Core 2.1 around sockets and http performance. On the client side, these new managed libraries are written from the ground up in managed code using the new high-performance Span<T> and on the server-side I could use Kestrel's (Kestrel is the .NET Core webserver) experimental UseSockets() as they are starting to move that over.

In other words, I can bypass libuv usage entirely by changing my Program.cs to use the use UseSockets() like this.

public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
     WebHost.CreateDefaultBuilder(args)
     .UseSockets()
     .UseStartup<Startup>();

Let's run it again. I'll add the ASPNETCORE_URLS environment variable and set it to a high port like 8080. Remember, I'm not admin so I can't use any port under 1024.

secretname@theirvmname [~/www]$ export ASPNETCORE_URLS="http://*:8080"
secretname@theirvmname [~/www]$ ./supercheapaspnetsite
Hosting environment: Production
Content root path: /home/secretname/public_html
Now listening on: http://0.0.0.0:8080
Application started. Press Ctrl+C to shut down.

Holy crap it actually started.

Ok, but I can't access it from supercheapaspnetsite.com:8080 because this is GoDaddy's locked down managed shared hosting. I can't just open a port or forward a port in their control panel.

But. They use Apache, and that has the .htaccess file!

Could I use mod_proxy and try this?

ProxyPassReverse / http://127.0.0.1:8080/

Looks like no, they haven't turned this on. Likely they don't want to proxy off to external domains, but it'd be nice if they allowed localhost. Bummer. So close.

Fine, I'll proxy the traffic myself. (Not perfect, but this is all a spike)

RewriteRule ^(.*)$  "show.php" [L]

Cool, now a cheesy proxy goes in show.php.

<?php
$site = 'http://127.0.0.1:8080';
$request = $_SERVER['REQUEST_URI'];

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $site . $request);
curl_setopt($ch, CURLOPT_HEADER, TRUE);
$f = fopen("headers.txt", "a");
    curl_setopt($ch, CURLOPT_VERBOSE, 0);
    curl_setopt($ch, CURLOPT_STDERR, $f);
    #don't output curl response, I need to strip the headers.
    #yes I know I can just CURLOPT_HEADER, false and all this 
    # goes away, but for testing we log headers
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$hold = curl_exec($ch);

#strip headers
$header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE);
$headers = substr($hold, 0, $header_size);
$response = substr($hold, $header_size);
$headerArray = explode(PHP_EOL, $headers);

echo $response; #echo ourselves. Yes I know curl can do this for us.
?>

Cheesy, yes. Works for GET? Also, yes. This really is Apache's job, not ours, but kudos to Tomas for this evil idea.

An ASP.NET Core app at a host that doesn't support it

Boom. How about another page at /about? Yes.

Another page with ASP.NET Core at a host that doesn't support it

Lovely. But I had to run the app myself. I have no supervisor or process manager (again this is already handled by GoDaddy for PHP but I'm in unprivileged world.) Shooshing in and running it is a bad idea and not sustainable. (Well, this whole thing is not sustainable, but still.)

We could copy "screen" over and start it up and detach like use screen ./supercheapaspnet app, but again, if it crashes, no one will start it. We do have crontab, so for now, we'll launch the app on a schedule occasionally to do a health check and if needed, keep it running. Also added a few debugging tools in ~/bin:

secretname@theirvmname [~/bin]$ ll
total 304
drwxrwxr-x  2    4096 Feb 28 20:13 ./
drwx--x--x 20    4096 Mar  1 01:32 ../
-rwxr-xr-x  1  150776 Feb 28 20:10 lsof*
-rwxr-xr-x  1   21816 Feb 28 20:13 nc*
-rwxr-xr-x  1  123360 Feb 28 20:07 netstat*

All in all, not that hard. ASP.NET Core and .NET Core underneath it can run pretty much anywhere, just like PHP, Python, whatever.

If you're a host and you want to talk to someone at Microsoft about setting up ASP.NET Core shared hosting, email Sourabh.Shirhatti@microsoft.com and talk to them! If you are GoDaddy, I apologize, and you should also email. ;)


Sponsor: Get the latest JetBrains Rider for debugging third-party .NET code, Smart Step Into, more debugger improvements, C# Interactive, new project wizard, and formatting code in columns.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb
Page 1 of 3 in the Linux category Next Page

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.