Scott Hanselman

Integrating Ruby and Watir with NUnit

June 30, '05 Comments [15] Posted in ASP.NET | Ruby | Watir | Javascript | NUnit | Bugs | Tools
Sponsored By

OK, I'm hooked. Ruby is the bomb and I'm loving it. We shall see what the future brings, the debugger is iffy, I need to recompile Notepad2 like Wes did with Ruby syntax support, but otherwise I'm digging it.

The back story is this. I've been trying to find the Holy Grail, or even the 40% Grail of Automated Web UI Testing for at least the last 5 years. We use NUnit for everything, so it'd be cool if whatever we chose could fail NUnit tests.

Lately folks have been trying again all over the next and lots of good stuff has been happening in this space.

  • NUnitASP - http://nunitasp.sourceforge.net/
    • Pros: C# class library for downloading and parsing web pages. Very CSharpy.
    • Cons: Nothing has happened with this since November 2004. Have to extend it constantly for more complex controls. Thinks about things in a Control way.
  • SAMIE - http://samie.sourceforge.net/
    • Pros: Uses Perl to drive IE around using it's OLE automation interface. Keeps logs, can fill with data from databases, and there's a cheesy but functional WinForms app to help write the scripting language. 
    • Cons: Uses Perl. LOL.
  • HttpUnit - http://httpunit.sourceforge.net/
    • Pros: Feels good to the Java folks, integrates with JUnit, lower level focused.
    • Cons: May not be actively developed, no activity since October 2004.
  • IEUnit - http://ieunit.sourceforge.net/
    • Pros: Almost zero installation footprint. Javascript based engine uses the language we were all born knowing. Rich debugging because you can use the Windows Script Debugger. Fairly simple in design. Actively developed. Nice WinForms script helper. Good if you already dig the DOM.
    • Cons: Slightly cheesy handling of model dialogs. Steals focus while running tests.
  • Selenium - http://selenium.thoughtworks.com/
    • Pros: Good for testing cross browser compatibility. Actively worked on. Selenium tests can be expanded to use other languages.
    • Cons: Uses a Fit-like HTML as it's source code. This takes a mind shift and freaks out some folks. Server-side component required.

And the one I'm currently enamored with is Watir. There's a great comparison of Watir and Selenium by a contributor to both here.:

  • Watir (Pronounced "Water") Web Application Testing in Ruby - http://wtr.rubyforge.org/
    • Pros: Actively being worked on. Uses Ruby, the language du jour. Can run tests asychronously. Simple, elegant, fast. Ruby's interactive shell makes it easy to develop tests (more on this later in this post). Great support for client side things like script events, and even mouse rollovers.
    • Cons: Haven't found any yet, but I'm sure they are out there. Very focused on the DOM.

Ok, just to level set if you made it this far: Watir is an open source functional testing library for automated tests to be developed and run against a web browser. Cool?

Here's an example that visits Google, searches for "Programming Ruby" and confirms that it found what it was looking for. The stuff after the # are comments.

require 'watir' # the watir controller
include Watir
test_site =
'http://www.google.com'
ie = IE
.new
ie.goto(test_site)
ie.text_field(:name, "q").set("pickaxe")
# q is the name of the search field
ie.button(:name, "btnG").click # "btnG" is the name of the Search button
if ie.contains_text("Programming Ruby")
  puts "Test Passed. Found the test string: 'Programming Ruby'. Actual Results match Expected Results."
else
  puts "Test Failed! Could not find: 'Programming Ruby'"
end

Very simple. So, Patrick and I wanted to integrate this into our NUnit tests, some of which already use Cassini for self-contained tests, and others are more complex but ultimately result in a web site. We wanted to do something like this from our existing infrastructure:

[TestFixture]
    public class WatirTest 
    {
        public WatirTest(){}
 
        [Test]
        public void FirstTest()
        {
            WatirAssert.TestPassed("ibcexample.rb");
        }
    }

This was the Goal. We talked about adding [WatirTest] attributes or extending a new WatirTestCase and even a whole new series of Assertions, but we didn't see any reason to be tricky, other than to be tricky. We opted for simple simple simple.

public class WatirAssert 
{
    public static void TestPassed(string rubyFileName)
    {
        string output = String.Empty;
        using(Process p = new Process())
        {
            p.StartInfo.UseShellExecute = false;
            p.StartInfo.RedirectStandardOutput = true;
            p.StartInfo.FileName = "ruby.exe";
            p.StartInfo.Arguments = rubyFileName;
            p.StartInfo.WorkingDirectory = AppDomain.CurrentDomain.BaseDirectory;
            p.Start();
            output = p.StandardOutput.ReadToEnd();
            p.WaitForExit();
        }
        Console.Write(output);
        Trace.Write(output);
        Regex reg = new Regex(@"(?<tests>\d+) tests, (?<assertions>\d+) assertions, 
(?<failures>\d+) failures, (?<errors>\d+) errors",
            RegexOptions.Compiled);
        Match m = reg.Match(output);
        try
        {
            int tests = int.Parse(m.Groups["tests"].Value);
            int assertions = int.Parse(m.Groups["assertions"].Value);
            int failures = int.Parse(m.Groups["failures"].Value);
            int errors = int.Parse(m.Groups["errors"].Value);
 
            if (tests > 0 && failures > 0) 
            {
                Assert.Fail(String.Format("WatirAssert: Failures {0}", failures));
            }
            else if (errors > 0) 
            {
                Assert.Fail(String.Format("WatirAssert: Errors {0}", errors));
            }
        }
        catch(Exception e)
        {
            Assert.Fail("WatirAssert EXCEPTION: " + e.ToString());
        }
    }
}

Using this, and armed with increasing knowledge of Ruby we started writing test cases (one per use case) for a large bank as a good test of this concept. These tests would run at the very end of the build process. After the build, test, docs, etc, we'd run the smoke (or integration) tests using Watir.

UPDATE: Your Ruby classes have to derive from the Ruby Unit Testing framework for this to work automatically. So your "template" would be (Note: the class name has an initial capital letter):

require 'watir'
include Watir
require 'test/unit'

class TestExampleTemplate < Test::Unit::TestCase 
  def test_search
    ie = IE.new
    ie.goto("
http://www.google.com")
    ie.text_field(:name, "q").set("pickaxe")
    ie.button(:value, "Google Search").click
    assert(ie.contains_text("Programming Ruby, 2nd Ed."))
  end
end

Now, referring back to the syntax:

ie.text_field(:name, "q").set("pickaxe") # q is the name of the search field
ie.button(:name, "btnG").click # "btnG" is the name of the Search button

You may think it'll be a hassle to do a lot of View|Source work and figure out the names and IDs of these textboxes and buttons. However, even though Watir only runs IE (remember it wraps IE's COM stuff) FireFox presented a great way to avoid all that poking through HTML source. Yes, it's the greatest thing FireFox brought to the Web Developer (besides Venkman), it's the Web Developer Extension.

Bring up your site in FireFox and from the Web Developer toolbar, click Forms|Display Form Details. Notice as in the screen shot below you'll get the IDs and Names of all your form elements. Then code your Watir script to them. Shiny.

Webdeveloperformsdisplay

If you want get Ruby and Watir and start playing:

  • Download Ruby for Windows
  • Download Watir and run install.rb
  • Go into C:\watir_bonus\examples and run googleSuite_test.rb
  • Then, look at articleExample.rb and make some changes to test your own site.
  • Enjoy. Feel free to integrate into your NUnit tests if you like using our code above. If you expand it, let us know.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Thursday, June 30, 2005 1:51:55 AM UTC
Hi Scott,
This looks great. I have worked a little on an Open Source project called IEUnit (via SourceForge) which does very similar things using javascript tests. It only runs on Windows, but has a lot of the same features, and additionally can create output in the same format that NUnit does, so the tests can be included in an NAnt script, and Nunt2Report can be used to provide a report in the same format as Nunit reports do.
There is debug support, I use VS.Net and run the scripts inside VS.Net.
Anyway, I am not trying to sell it, just mention it to see how people find it in comparison to other projects.
I will be trying Watir too, thanks form mentioning it.
Mark Focas
Thursday, June 30, 2005 5:55:04 AM UTC
Glad to see you finally drank the Ruby kool-aid. Have fun :).
Thursday, June 30, 2005 6:45:46 AM UTC
Mark - Crap, I knew I forgot IEUnit. I spent time with that one too. I will add it to the post. I was trying to remember which one I forgot in my travels. Thanks!
Thursday, June 30, 2005 12:31:58 PM UTC
Ruby rocks! :) you see they posted the results for Rails Day www.railsday.com ... check it out!
Thursday, June 30, 2005 2:48:08 PM UTC
Oh my gosh! Where are the {} and ;'s??? Are you kidding? This can't be a real language. Looks too much like VB...
NoSemiColon
Thursday, June 30, 2005 3:28:11 PM UTC
"I need to recompile Notepad2 like Wes did with Ruby syntax support"

Make sure you outline how you did this, unlike Mr. Moise.
Matt T.
Thursday, June 30, 2005 4:44:42 PM UTC
I'm curious why "Uses a Fit-like HTML as it's source code." is a con for Selenium. I guess it depends on what you're trying to do. Selenium is perfect for me because i'm looking for a good way to automate acceptance tests through my webapp. My tester doesn't know programming languages, but she can figure out Fit tables (actually I integrated it with Fitnesse, so all she needs to know is how to use a wiki). To me, that's a pro, rather than a con. But if I'm looking to do developer written integration tests, I can see how this might not be ideal (versioning, source control, etc).
Andy O
Friday, July 01, 2005 4:26:40 AM UTC
Andy, from my point of view, writing this...

|eg.Division|
|numerator|denominator|quotient?|
|10 |2 |5 |
|12.6 |3 |4.2 |
|100 |4 |33 |

...is pretty asking your tester to write something in an obscure language. At least Watir/Ruby gives us good compiler-like syntax checking, near infinite flexibilty, and using Watir::Simple (an even SIMPLER wrapper on Watir) is as easy as writing HTML.
Friday, July 01, 2005 2:00:57 PM UTC
I'm in the same situation, and have also discovered Watir. I've been half-heartedly trying to integrate it into Fitnesse so that our testers can define their acceptance tests without bothering us to write the tests. Watir is our choice because the web app is internal and uses Integrated Authentication.

I must say I love the NUnit integration!! I can't wait to try it!
Kevin Williams
Monday, July 25, 2005 4:03:01 PM UTC
Hi,
very cool. Have you come across SWEA? check it out on http://www.alexfurman.net/

According to the blurb on the site
What is SWExplorerAutomation?
The program creates an automation API for any Web application which uses HTML and DHTML and works with Microsoft Internet Explorer. The Web application becomes programmatically accessible from any .NET language.
SWExplorerAutomation API provides access to Web application controls and content. The API is generated using SWExplorerAutomation Visual Designer. SWExplorerAutomation Visual Designer helps create programmable objects from Web page content.


cheers
benjy
Thursday, August 18, 2005 4:34:07 PM UTC

I am extremely new to ruby, so bear with me.

How do you script a LinkButton's postback? Below is an example output of what the LinkButton is rendered as.

&lt;a href="__doPostBack('_ctl2','')"&gt;&lt;font face="Verdana" size="4"&gt;Click Me!&lt;/font&gt;&lt;/a&gt;
andrew
Thursday, August 18, 2005 4:42:31 PM UTC

Fyi, here's how it is done. Replace XYZ with javascript (cannot submit the text as is due to .net security).

```````````


require 'watir'
include Watir
require 'test/unit'
class TC_recorded < Test::Unit::TestCase
def test_recorded
ie = IE.new
ie.goto('http://samples.gotdotnet.com/quickstart/aspplus/samples/webforms/ctrlref/webctrl/LinkButton/VB/LinkButton1.aspx')
ie.link(:url, "XYZ:__doPostBack('_ctl1','')").click
end
end
andrew
Sunday, September 04, 2005 3:51:16 AM UTC
Thanks for mentioning IeUnit in your review.
Just a minor clarification in the comment about IeUnit: I guess 'model dialog'
should be 'modal dialog'. IeUnit supports handling of modal dialogs with
integrated Win32Dom object which is not as easy as the DHTML/DOM access.

Some comments about WatirMaker:
During the development of IeUnit we also went to the path to develop some kind of
record/play utility. But we soon noticed that this tool will change the 'gravity'
of the software: it would have required too many changes in the current code base
to archieve a consistent solution. We then ended up with a tool that is
basically an inspection tool that allows programmers to quickly identify
HTML objects. I still think a record/play tool is doable and useful, but
this will significantly change of the focus of the software.

Saturday, October 22, 2005 10:06:45 PM UTC
First of all, thank's for a great article!

I've been playin around with Watir and the NUnit code that you supplied, and I think it's great. However, since I'm a total newbie at Ruby I have a question for you. How do you supply the Ruby/Watir scripts with configuration data? I'm using this in conjuction with NUnitASP, and I have been using AppSettings in a regular config file for the config data. Do you have any comments or solutions for this? Where do you store the config data for your scripts? Would be very greatful if you could shed some light on this.
Tuesday, November 08, 2005 6:22:28 PM UTC
I am using Selenium at the moment, and I like it. However, in my mission to eradicate typing from the world I was chuffed to find the Selenium Recorder http://seleniumrecorder.mozdev.org/ which is a "Firefox extention that captures the input from user, and generates Selenium commands".

Good to generate a starter for 10 test that you can then modify with asserts etc (or not if you are super lazy boy). It's a bit basic right now but I have high hopes...
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.