Scott Hanselman

An Xml Tidy in PowerShell or Formatting Xml with Indenting with PowerShell

July 3, '06 Comments [4] Posted in PowerShell | XML
Sponsored By


I like my XML pretty. There's no format-xml cmdlet or tidy-xml in PowerShell, so here's my first try:

#Name me tidy-xml.ps1
# - this crap written by Scott Hanselman
[System.Reflection.Assembly]::LoadWithPartialName("System.Xml") > $null
$PRIVATE:tempString = ""
if ($args[0].GetType().Name -eq "XmlDocument")
{
 $PRIVATE:tempString = $args[0].get_outerXml()
}
if ($args[0].GetType().Name -eq "String")
{
 $PRIVATE:tempString = $args[0]
}
$r = new-object System.Xml.XmlTextReader(new-object System.IO.StringReader($PRIVATE:tempString))
$sw = new-object System.IO.StringWriter
$w = new-object System.Xml.XmlTextWriter($sw)
$w.Formatting = [System.Xml.Formatting]::Indented
do { $w.WriteNode($r, $false) } while ($r.Read())
$w.Close()
$r.Close()
$sw.ToString()

Sometimes XML is thought of as strings and sometimes as [xml] in PowerShell. This script will take either a string or [xml] but will always return a string. (e.g. It's on you to do the final [xml] cast because if you did, the tidying is moot). For example:

PS> $a = "<foo><bar>asdasd</bar></foo>"
PS> ./tidy-xml $a
<foo>
  <bar>asdasd</bar>
</foo>
PS> $b = [xml]"<foo><bar>asdasd</bar></foo>"
PS> ./tidy-xml $b
<foo>
  <bar>asdasd</bar>
</foo>

I wanted to make it so I could do these scenarios. Thoughts? Remember that I need to normalize to a string for the StringReader constructor.

#couldn't because it returned an Object[] of strings and it got sloppy fast
get-content foo.xml | tidy-xml

#couldn't because it (oddly) returned an ArrayList of strings and it got sloppy fast
get-content foo.xml -ov c
tidy-xml $c

Enjoy (or improve!)

UPDATE: Here's a better version that includes a number of best-practices changes as well as the support for taking IN objects from the pipeline (like I wanted originally):

#The following cases work
#
#PS>$a
#<foo><bar>this is A</bar></foo
#PS>$b.get_OuterXml()
#<foo><bar>this is B</bar></foo
#PS>Get-Content foo.xml
#<foo>
#   <bar>this is C</bar>
#</foo>
#
#Now try the following.
#PS>sal ti tidy-xml
#PS>$a | ti
#PS>$b | ti
#PS>$c | ti
#PS>ti $a
#PS>ti $b
#PS>ti $c
#PS>$a, $b | ti
#PS>$a, $c | ti
#PS>$c, $b | ti
#PS>$a, $b, $c | ti
#
#What doesn't work here is when you pass a multiple parameter input as follows:
#tidy-xml $a, $b # doesn't work
#
#Uhm, i think i would have to change my logic "completely" to actually get that to work...
#(after refactoring "process" block...)
#
#Name me tidy-xml.ps1
# - some of this crap written by Scott Hanselman
function Tidy-Xml {
    begin {
        $private:str = ""
       
        # recursively concatenate strings from passed-in arrays of schmutz
        # not sure how to improve this...
        function ConcatString ([object[]] $szArray) {
            # return string
            $private:rStr = ""

            # Recursively call itself, if a string is also of array or a collection type
            foreach ($private:sz in $szArray) {
                if (($private:sz.GetType().IsArray) -or `
                    ($private:sz -is [System.Collections.IList])) {
                    $private:rStr += ConcatString($private:sz)
                }
                elseif ($private:sz -is [xml]) {
                    $private:rStr += $private:sz.Get_OuterXml()
                }
                else {
                    $private:rStr += $private:sz
                }
            }
            return $private:rStr;
        }
       
        # Original "Tidy-Xml" portion
        function FormatXmlString ($arg) {
            # ignore parse errors
            trap { continue; }
           
            # out-null hides output of the assembly load
            [System.Reflection.Assembly]::LoadWithPartialName("System.Xml") | out-null

            $PRIVATE:tempString = ""
            if ($arg -is [xml]){
                $PRIVATE:tempString = $arg.get_outerXml()
            }
            if ($arg -is [string]){
                $PRIVATE:tempString = $arg
            }

            # the ` tick mark is a line-continuation char
            $r = new-object System.Xml.XmlTextReader(`
                new-object System.IO.StringReader($PRIVATE:tempString))
            $sw = new-object System.IO.StringWriter
            $w = new-object System.Xml.XmlTextWriter($sw)
            $w.Formatting = [System.Xml.Formatting]::Indented

            do { $w.WriteNode($r, $false) } while ($r.Read())

            $w.Close()
            $r.Close()
            $sw.ToString()
        }
    }
   
    process {
        # For non-xml strings or types, they will be buffered and will be
        # taken care of in "end" block
        
        # this checks for objects that have been "pipe'd" in.
        if ($_) {
            # check if whatever we have appended is a valid XML or not
            $private:xmlStr = ($private:str + $_) -as [xml]
           
            if ($private:xmlStr -ne $null) {
                FormatXmlString([xml]$private:xmlStr)
                # clear the string not to be handled in "end" block
                $private:str = $null
            } else {
                if ($_ -is [string]) {
                    $private:str += $_
                } elseif ($_ -is [xml]) {
                    FormatXmlString($_)
                }
                # for an array or a collection type,
                elseif ($_.Count) {
                    # iterate each item in the collection and append
                    foreach ($i in $_) {
                        $private:line += $i
                    }
                    $private:str += $private:line
                }
            }
        }
    }

    end {
        if ([string]::IsNullOrEmpty($private:str)) {
            $private:szXml = $(ConcatString($args)) -as [xml]
            if (! [string]::IsNullOrEmpty($private:szXml)) {
                FormatXmlString([xml]$private:szXml)
            }
        } else {
            FormatXmlString([xml]$private:str)
        }
    }
}

Thanks to MonadBlog for the Updates! There's definitely some room for refactoring of the begin/end/process, but it's more funcitonal this way.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Querying Virtual Server 2005 via VM with PowerShell

July 3, '06 Comments [4] Posted in PowerShell
Sponsored By

One of the guys in IT manages a lot of Virtual Server instances, like dozens, adding up into many dozens of Virtual Machines all supporting our many devs. He wanted to get some status information with PowerShell. Here's what I came up with.

We used WMI Explorer to check out the WMI namespace installed by Virtual Server (root/vm/virtualserver).

Given a CSV file like this full of (at least) Virtual Server ComputerName

computername,owner,whatever
MSVS1,fred,somedata
MSVS2,joe,somedata
MSVS3,luigi,somedata

We did this:

import-csv servers.csv | foreach-object
{  $_.computername   } |
foreach-object
{ Get-WmiObject -computername $_ -namespace "root/vm/virtualserver" -class VirtualMachine } |
select 
@{Expression={"__SERVER"}; Name="Server"},
Name, CpuUtilization,
select @{Expression={"Uptime/60"}; Name="Uptime(min)"},
PhysicalMemoryAllocated,
DiskSpaceUsed |
format-table -groupby Server -property name,CpuUtilization,Uptime,PhysicalMemoryAllocated,DiskSpaceUsed

Which gives us more or less this:

Vsserver1

Which can also reformat, make smaller and run in a loop to get a "top" equivalent for all our VMs. We can catch machines that are running out of space, working too hard, and do capacity planning.

Note, I originally wanted to do this:

import-csv servers.csv | Get-WmiObject -namespace "root/vm/virtualserver" -class VirtualMachine

and have "computername" automatically bound to because the name was the same in the CSV file and the powershell parameter name. This does work in this instance...make a CSV file like this named pids.csv:

ID
1
2
3
4
5

and execute this PowerShell pipeline

import-csv pids.csv | get-process

and you'll get

Get-Process : No process with process ID 1 was found.
At line:1 char:33
+ import-csv pids.csv | get-process <<<<
Get-Process : No process with process ID 2 was found.
At line:1 char:33
+ import-csv pids.csv | get-process <<<<
Get-Process : No process with process ID 3 was found.
At line:1 char:33
+ import-csv pids.csv | get-process <<<<

Handles  NPM(K)    PM(K)      WS(K) VM(M)   CPU(s)     Id ProcessName
-------  ------    -----      ----- -----   ------     -- -----------
   2399       0        0         32     2   507.75      4 System
Get-Process : No process with process ID 5 was found.
At line:1 char:33
+ import-csv pids.csv | get-process <<<<

See how it called get-process for each ID and it automatically bound the ID column of the table coming from the CSV to the ID property? I wanted to do the same with with computername, but it didn't work.

Get-WmiObject : The input object cannot be bound to any parameters for the comm
and either because the command does not take pipeline input or the input and it
s properties do not match any of the parameters that take pipeline input.
At line:1 char:21

I got this error which I assume means that Get-WMIObject doesn't take pipeline input in the build of PowerShell I have (RC1). I hope this is queued to get fixed ASAP or I'm just missing something.

UPDATE: A "help get-wmiobject" (duh, RTFM) confirms that -computername doesn't take pipeline input.

    -ComputerName <System.String[]>
        Declares on which computer(s) the WMI object may be found

        Parameter required?           false
        Parameter position?           named
        Parameter type                System.String[]
        Default value                 localhost
        Accept multiple values?       true
        Accepts pipeline input?       false
        Accepts wildcard characters?  false

Bummer.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Searching Google Desktop from PowerShell

July 2, '06 Comments [1] Posted in PowerShell | Javascript | XML
Sponsored By

I just noticed that Sean McLeod created a nice cmdlet for PowerShell that lets you query Windows Desktop Search from the PowerShell Command Line. I used to use Windows Desktop Search but got tired of it hanging and switched to, and settled on, for now, Google Desktop Search and I've been very happy with it.

After reading this I said, I'd like to created a cmdlet for Google Desktop Search, but who has the time? (Maybe I'll do it later)

I figured I could do something quick and dirty though. I took a look at the Google Desktop API Developer's Guide and saw some JavaScript samples for their psycho and obscure COM API. (Google does nearly everything in C++. Even their .NET sample code smells like C++.)

I did this at the PowerShell command line, as a client of the Google Desktop Search has to "register" itself, and I got stuck:

PS>$registrar = new-object -com "GoogleDesktop.Registrar"
PS>$regArray = "Title","Searching GDS from PowerShell","Descripton","For Fun","Icon","My
Icon@1"
PS>$regId = [System.Guid]::NewGuid().ToString("B").ToUpper()
PS>$registrar.StartComponentRegistration($regId,$regArray)
Exception calling "StartComponentRegistration" with "2" argument(s): "The component description must contain a SAFEARRAY of six or eight VARIANTs" At line:1 char:38 + $registrar.StartComponentRegistration( <<<< $regId,$regArray)

...and they lost me at SAFEARRAY. I don't know why this little .NET array didn't get marshalled correctly. Probably because it's marshalled as a SAFEARRAY of BSTRs instead. Anyway, too hard, patience waning.

But, hey, GDS runs a local web server, right? So I should be able to query it locally. The docs say:

The search query URL, including your security token, is stored in the registry at:
HKEY_CURRENT_USER\Software\Google\Google Desktop\API\search_url
To use the example above, the stored query URL would be something like:
http://127.0.0.1:4664/search&s=1ftR7c_hVZKYvuYS-RWnFHk91Z0?q=

So from PowerShell:

PS>(Get-Item "HKCU:\Software\Google\Google Desktop\api").GetValue("search_url")
http://127.0.0.1:4664/search&s=1ftR7c_hVZKYvuYS-RWnFHk91Z0?q=

Cool. Then I can add the query after the "q=" and "&format=xml" to get something that PowerShell can sink its teeth into. I'll need to UrlEncode the query.

UPDATE: Note the num= and flags= at the end of the queryString. That indicates that we are only interested in files and we'll take as many as 1000.

PS>$query = "PowerShell"
PS>$searchUrl = (Get-Item "HKCU:\Software\Google\Google Desktop\api").GetValue("search_url")
PS>[System.Reflection.Assembly]::LoadWithPartialName("System.Web") > $null
PS>$newQuery = $searchUrl + [System.Web.HttpUtility]::UrlEncode($query) + "&format=xml&flags=576&num=1000"
PS>$webclient = new-object System.Net.WebClient
PS>$resultsXml = [xml]($webclient.DownloadString($newQuery))
PS>$resultsXml.results.result | select title, snippet, category

title                      snippet                    category
-----                      -------                    --------
C:\Temp\fusionlogs\Defa... NET\Framework\v2.0.5072... file
Windows <b>PowerShell</... Windows <b>PowerShell</... web
del.icio.us/shanselman     del.icio.us/shanselman ... web
C:\Temp\fusionlogs\Defa... NET\Framework\v2.0.5072... file
Windows Desktop Search ... Windows Desktop Search ... web
User Profile: Greg Borota  User Profile: Greg Boro... web
del.icio.us/shanselman     del.icio.us/shanselman ... web
Microsoft Survey            Microsoft Survey Th... web
Discussions in Windows ... Discussions in Windows ... web
Search Results: <b>powe... Search Results: <b>powe... web

Cool. So now I want just the files, and I want them to be actual FileInfo objects.

PS> $resultsXml.results.result | where { $_.category -eq "file"} | foreach-object { get-item $_.url }

    Directory: Microsoft.PowerShell.Core\FileSystem:: C:\Temp\fusionlogs\Default\powershell.exe

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---          7/2/2006   3:08 AM       1625 System.Web.HTM
-a---          7/2/2006   2:36 AM       1695 System.Windows.Forms.HTM

...etc...

Now I'll save the whole file in my path in a file called "search-googledesktop.ps1" and change the first line to "$query = $args" to take command line arguments.

Since the objects that are coming out of this script are real FileInfo objects retrieved with the call to get-item in the final line, I can use my script in a larger pipeline like this:

.\search-googledesktop.ps1 "powershell" | get-content

or

.\search-googledesktop.ps1 "powershell" | get-location

or delete any file that has my personal information like a social security number in it.

.\search-googledesktop.ps1 "123" | remove-item -whatif

Seriously, once you have an indexer on your system, search for your Social Security Number. You'd be amazed at the old Excel sheets and crap that number gets into.

Anyway, cool. Sure beats their crap COM API.

File Attachment: search-googledesktop.ps1 (474 bytes)

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Using FFMPEG to squish lots of videos using PowerShell

July 2, '06 Comments [2] Posted in PowerShell | XML | Gaming | Tools
Sponsored By

This post will likely win the award this month for "least useful and most obscure use of two totally unrelated tools to do a task that few folks ever would need to do and fewer would be interested in reading about" but I wanted to save it for myself for reference.

I've got a bunch of old TV shows that are in MPEG2 format that were recorded with BeyondTV from SnapStream. I don't want to toss them, but they are fat and wasting space. I'd like to have them available if I wanted to watch them on my PlayStation Portable, but using PSPVideo9 is tedious for more than a few files. Also, the version of FFMPEG that comes with PSPVideo9 is really old.

You can get a binary copy of FFMPEG (as the project team releases source but not binaries) from this friendly fellow here.

FFmpeg is a collection of free software that can record, convert and stream digital audio and video. It includes libavcodec, a leading audio/video codec library. FFmpeg is developed under Linux, but it can be compiled under most operating systems, including Windows. The project was started by Gerard Lantau, an alterego of Fabrice Bellard, and is now maintained by Michael Niedermayer. Notable is that most FFmpeg developers are part of either the MPlayer, xine or VideoLAN project as well. [Wikipedia]

FFmpeg doesn't like WMV files very much, and they are compressed pretty well already. I wanted to run this ffmpeg command line...

ffmpeg -i 'oldname.*' -f mp4 -r 25 -s 320×240 -b 768 -ar 44000 -ab 112 'newname.mp4'

...on all my files that weren't WMV and convert my files to MP4 video at 320x240 and a decent bit rate.

I used PowerShell to exclude this command line, build the filenames and ran this PowerShell command (pipeline) on my folder:

dir -exclude *.wmv | foreach-object { $newname = $_.Basename + ".mp4"; ffmpeg -i "$_" -f mp4 -r 25 -s 320×240 -b 768 -ar 44000 -ab 112 $newname }

Ran for a few hours and the PlayStation Portable is now happy with my copies of old TV shows that aren't on DVD yet.

NOTE: The System.IO.FileInfo object, returned from PowerShell's 'dir', doesn't have a "BaseName" property, so I've added a custom ScriptProperty to my "My.Types.ps1xml" file:

<Types>
   <Type>
     <Name>System.IO.FileInfo</Name>
      <Members>
         <ScriptProperty>
           <Name>Basename</Name>
           <GetScriptBlock>
            $this.Name.Remove($this.Name.Length - $this.Extension.Length);
           </GetScriptBlock>
         </ScriptProperty>
      </Members>
   </Type>
</Types>

As outlined in this post called "BaseName for FileInfo objects" on the PowerShell team blog.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Sapphire In Steel - Ruby within Visual Studio 2005

July 2, '06 Comments [0] Posted in PowerShell | Ruby | Watir | Bugs
Sponsored By

Steel_ideWell, apparently I'm the last to notice this one (likely because I've been up to my ears in PowerShell, not Ruby) but "SapphireInSteel" is a Visual Studio 2005 Project add-on that adds Ruby *editing* support to Visual Studio.

This has nothing to do with turning Ruby into IL or making Ruby a .NET Language (although those projects exist.) This is about editing and debugging Ruby within Visual Studio 2005.

If you haven't played with Ruby, take a moment and do this:

This'll be great for everyone who does both Ruby and .NET, but especially for those doing .NET plus Watir development. This is apparently a very actively development project so subscribe to their blog.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by SherWeb

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.