Scott Hanselman

NuGet Package of the Week: FluentAutomation for automated testing of Web Applications

March 4, '14 Comments [25] Posted in NuGet | NuGetPOW | Open Source
Sponsored By

FluentAutomation starting a testLast week I was exploring today's varied choices we have for Automated Browser Testing. There's headless WebKit "browsers" like PhantomJS and cloud powered multi-browser testing tools like BrowserStack and SauceLabs.

Selenium is kind of the gold standard and offers not only a lot of "drivers" but also a lot of language bindings with which drive a browser. Sometimes browsers update so fast there can be some version incompatibilities with Selenium, but for the most part it works great once you've settled in.

One option I've been looking at is FluentAutomation. It's a fluent automation API that supports Selenium as well as WatiN along with all their flavors and drivers. Since Fluient supports Selenium, that means you can use the Selenium ChromeDriver, IEDriver, Remote Web Driver or even the headless PhantomJS. FluentAutomation is on GitHub, of course, as well as on NuGet.

FluentAutomation has great (and growing) documentation and has adopted and interesting fluent style for it's API.

Now, not everyone likes a "fluent" API so it may take a while to get used to. Often you'll be doing things over many lines when it's really just one line, for example, this is one line:

I.Open("http://automation.apphb.com/forms")
.Select("Motorcycles").From(".liveExample tr select:eq(0)")
.Select(2).From(".liveExample tr select:eq(1)")
.Enter(6).In(".liveExample td.quantity input:eq(0)")
.Expect
.Text("$197.72").In(".liveExample tr span:eq(1)")
.Value(6).In(".liveExample td.quantity input:eq(0)");

Notice the method chaining as well as the use of CSS selectors.

FluentAutomation also has the cool concept of a PageObject to take your potentially brittle scripts and give them more structure. PageObjects group your actions, expectations, and assertions and let you reuse code when a page appears in multiple tests.

For example you could have a high level test (this is XUnit, but you can use whatever you want):

public class SampleTest : FluentTest {
public SampleTest() {
SeleniumWebDriver.Bootstrap(SeleniumWebDriver.Browser.Chrome);
}

[Fact]
public void SearchForFluentAutomation() {
new BingSearchPage(this)
.Go()
.Search("FluentAutomation")
.FindResultUrl("http://fluent.stirno.com/blog/FluentAutomation-scriptcs/");
}
}

Then you can have separate PageObjects that have your own public methods specific to that page, as well as assertions you can reuse.

public class BingSearchPage : PageObject<BingSearchPage> {
public BingSearchPage(FluentTest test) : base(test) {
Url = "http://bing.com/";
At = () => I.Expect.Exists(SearchInput);
}

public BingSearchResultsPage Search(string searchText) {
I.Enter(searchText).In(SearchInput);
I.Press("{ENTER}");
return this.Switch<BingSearchResultsPage>();
}

private const string SearchInput = "input[title='Enter your search term']";
}

public class BingSearchResultsPage : PageObject<BingSearchResultsPage> {
public BingSearchResultsPage(FluentTest test) : base(test) {
At = () => I.Expect.Exists(SearchResultsContainer);
}

public BingSearchResultsPage FindResultUrl(string url) {
I.Expect.Exists(string.Format(ResultUrlLink, url));
return this;
}

private const string SearchResultsContainer = "#b_results";
private const string ResultUrlLink = "a[href='{0}']";
}

You don't have to be all structure and OO if you don't want. You can just as easily write scripts with FluentAutomation and head in a different direction.

FluentAutomation along with ScriptCS = Automating your Browser with C# Script

I've usually used Python with my Selenium scripts. I like being able to just make a text file and start scripting, then run, debug, continue, all from the command line. It feels simple and lightweight. Creating a DLL and running Unit Tests in C# usually comes later, as I can move faster with a "scripting language."

You can do that with ScriptsCS as it gives you project-less C# that effectively is C# as scripting language. Combine this with FluentAutomation and you've potentially got the best of both worlds.

To install, first you need the Windows apt-get open source equivalent, the oddly-named and -spelled Chocolatey. Then you get ScriptCS and the packages for FluentAutomation.

  • Install Chocolatey - one line installation here
  • Run "cinst ScriptCS" from your command line to use Chocolatey to install ScriptCS
  • Now, get the ScriptCS script packages for FluentAutomation like this:
    • scriptcs -install FluentAutomation.SeleniumWebDriver
    • scriptcs -install ScriptCs.FluentAutomation

Now, as a quick test, create a folder and put a text file called start.csx in it with just these contents:

var Test = Require<F14N>()
.Init<FluentAutomation.SeleniumWebDriver>()
.Bootstrap("Chrome")
.Config(settings => {
// Easy access to FluentAutomation.Settings values
settings.DefaultWaitUntilTimeout = TimeSpan.FromSeconds(1);
});

Test.Run("Hello Google", I => {
I.Open(http://google.com);
});

Notice how there's no namespace, no classes, no main. It's just a script, except it's using C#. You can change the "Chrome" to "IE" or "Firefox" as well, to play around.

Random: I love this Selenium feature, exposed by FluentAutomation...take screenshot!

// Take Screenshot
I.TakeScreenshot("LoginScreen");

If you don't want ScriptCS, while it can act as a REPL itself, there is also the start of a dedicated FluentAutomation REPL (read–eval–print loop). This is basically a command prompt that lets you explore you app interactively and facilitates building your scripts. You can get the Repl as a Chocolatey package as well and just "cinst FluentAutomation.Repl"

You've got LOTS of choices in the world of automated testing. There's so many choices that there's just no good excuse. Pick a library, pick a language, and start automating your web app today.

Related Links


Sponsor: Big thanks to ComponentOne, a division of GrapeCity, for sponsoring the blog this week. Their widely popular .NET control suite, Studio Enterprise contains hundreds of data and UI controls such as grids, charts and reports that offer the functionality, features and support you need for current and future application development. Download your trial today!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Tuesday, March 04, 2014 2:47:41 AM UTC
Looks like a nice tools, although in reality, abstractions like these don't hold long in advanced scenario. I recently used Coypu which is a similar abstraction (less "abstract") that provides things like automatic retires and timeouts, etc, and I still find myself missing stuff and reaching out to core Selenium sometimes.

Apart from this, and more importantly:

FluentAutomation also has the unique concept of a PageObject


That's quite invalid.

PageObjects as a pattern is mentioned in Selenium's documentation

https://code.google.com/p/selenium/wiki/PageObjects

Also, mentioned by Martin Fowler:

http://martinfowler.com/bliki/PageObject.html

And utilized in other frameworks that build on top of Selenium, like:

http://docs.teststack.net/Seleno/PageObjects.html
Tuesday, March 04, 2014 2:56:52 AM UTC
I'm obviously biased, but I feel that any testing framework that relies on magic strings for locating elements is inherently flawed. I've been down that road, it ends with tests that become a barrier to change, not an enabler of change. Page objects and other patterns for encapsulating those strings only sweep the problem under the rug.

As an alternative, I created SpecsFor.Mvc. Automated hosting of your ASP.NET MVC app via IIS Express, streamlined wrapper on top of Selenium, integrated SMTP host for testing E-mails, and a strongly-typed, refactor-friendly API for interacting with your app. It's far from perfect; documentation is lacking (completely my fault), and there are still a lot of pain points it doesn't address, but I do feel it leads to much better tests than the string-based alternatives.

Tuesday, March 04, 2014 4:11:37 AM UTC
Thanks Mo! My mistake. I will update!

Matt - isn't HTML and all IDed elements magic strings?
Scott Hanselman
Tuesday, March 04, 2014 6:30:59 AM UTC
Thank you Scott!

I love this tool and think it deserves some more center-stage time. Been pushing it for my clients for some time, and now with the inclusion of the PageObject construct built-in as a core feature it took yet another step.

On behalf of the FluentAutomation-users out there I thank you for lifting this up to more peoples attention.

Keep your great blogging up!
Tuesday, March 04, 2014 7:06:43 AM UTC
From a readability point of view Fluent Assertions are great. However i cant help but notice that FluentAutomation breaks with the Arrange/Act/Assert (given/when/then) pattern. Setup, dataextraction and assertion is muddled into each other.

Has anyone tried using it with Cucumber/Specflow with any success?
Michael
Tuesday, March 04, 2014 8:14:48 AM UTC
And now we need the debugger to understand fluent APIs and let you step over method calls one by one...
Tuesday, March 04, 2014 10:04:30 AM UTC
@Michael. Yes, We use SpecFlow with FluentAutomation, but all the fluent side of things is pushed down into the page object methods. The steps definitions don't use it, they simply call down the page object to perform an action.

In theory this means changes to the UI are insulated from the feature file and the step definitions. The only changes are in the page object (which is were all the magic strings are used).
Mike Thomas
Tuesday, March 04, 2014 12:14:16 PM UTC
Scott: Not entirely, no. When done properly, ASP.NET MVC makes heavy use of conventions. Views are bound to strongly-typed view models, and strongly-typed helpers (like Html.EditorFor(m => m.Whatever)) create markup that fits conventions that the model binder also understands. URLs can also be generated using strongly-typed expressions (Html.ActionLink<HomeController>(c => c.Hello("World")). Yes, your view will still contain HTML, but most of the important bits should be generated by strongly-typed helpers.

SpecsFor.Mvc leverages those same conventions. You can specify the a strongly-typed lambda expression to locate and interact with an element on the page based on a view model (SUT.FindFormFor<RegisterModel>().Field(m => m.Email)), or use a strongly-typed lambda expression to navigate to the URL corresponding to an action on a controller (SUT.NavigateTo<AccountController>(c => c.Register())). Because your view and your specs are using strongly-typed expressions, you get compile-time safety and can refactor freely.

There will still be times when you have to drop down and do something with strings. Like I said, SpecsFor.Mvc isn't perfect. :) But those times should be the exception, not the norm.
Tuesday, March 04, 2014 1:52:24 PM UTC
I'm not very familiar with automated Gui testing so I have some (maybe silly) questions:

  1. What would exactly be the development scenario that FluentAutomation would give a test failure other then dead links?
  2. As far as I can see it does not fail on javascript error's for instance, unless some html is not generated due to Javascript errors.
  3. Also it would not detect if the page would render in purple instead of the desired pink.
  4. And does FluentAutomation also function where most / all of the html is generated from a Javascript framework?

Peter
Tuesday, March 04, 2014 2:29:52 PM UTC
Bertrand - Not sure I follow your comment (as compared to using Se directly) but would love more feedback. Thanks!

Michael - Lots of our users work with SpecFlow successfully using similar patterns as Mike Thomas. As for AAA, the chaining works in a way to encourage this. For example a single chain cannot expect/assert and then go back into an action. The arrange basically boils down to the Open command (or.. Given I'm on the Login Page), take some actions, assert success or fail conditions.

Matt - SpecsFor.Mvc is a very interesting project. I contemplated building something with a similar method of integrating with the app but wanted to maintain the ability to test *any* web app, not just ones built in our chosen stack. Perhaps a more general lib just capable of retrieving the appropriate selectors from MVC apps would help fill the gap. Interesting times we're in!
Tuesday, March 04, 2014 2:35:55 PM UTC
Oops. missed one..

Peter -

1. Test failures occur either because an action cannot successfully be taken (broken functionality in page, missing elements, etc) or because you've explicitly built in assertions, like any other code-based test.

2. There are many many reasons a javascript error might be thrown on a page that doesn't directly affect the functionality of the app, so we don't blow up on them.

3. If you have a known class for the page color, you can use I.Assert.Class("pink").On("body"); or something similar and get a failure for that occurrence. Asserts on inline styles coming soon.

4. Because it uses either Selenium or WatiN, FluentAutomation is working with the DOM in whatever state it happens to be in, regardless of where the elements came from. We have users testing giant Single-Page-App deployments, big MVC sites and all the way to Rails apps - the entire spectrum really.
Tuesday, March 04, 2014 2:57:33 PM UTC
Michael,

To me, this seems like more of a BDD-style testing framework, more than a TDD... and from what I've seen, in BDD style testing, the AAA model doesn't quite fit, since there are usually too many things happening in an order.

I used Jester to perform some BDD testing in PowerShell, and found the same thing... lots of steps, validation occurring between the steps, etc... all to validate that the process works end-to-end.
Tuesday, March 04, 2014 5:55:21 PM UTC
@Brandon

Great job on the library.

But I think Peter's point about an assertion that no error happened would be pretty useful. I agree with you that we don't want to fail every test, but an Assert.NoErrors() would be great. Heck, even an Assert.NoWarnings() could be useful.

Can't wait for the inline style assertions.

Keep up the good work.
Tuesday, March 04, 2014 11:01:55 PM UTC
Looks awesome. It's actually very similar to Nightwatch.js which I tried a couple of weeks ago. It's based on Node.js and Selenium server:
http://juristr.com/blog/2014/02/nightwatch-test-automation/
Thursday, March 06, 2014 3:49:41 AM UTC
If you're looking for a great combination of BDD/SpecFlow and Selenium check out SpecBind http://www.specbind.net . It combines easy to use speflow steps with a page model. It also supports CodedUI and is open source :)
Dan
Thursday, March 06, 2014 9:26:37 PM UTC
I think this is great!

And thanks for the blog post on Chocolatey / ScriptCS. I'm really hoping to see a scripting "culture" on Windows that's equatable with what's on Linux systems someday. It's so fun and powerful to be able to write scripts that are doing things other than server admin / command-line interfacing without having to open Visual Studio and create a project :)

Maybe instead of "scripting" language we should say "stand-alone source-code executable".
Tuesday, March 11, 2014 1:08:57 AM UTC
@Michael regarding the Except vs Assertion. You will be happy to note that the latest version of FluentAutomation supports both styles and that the more commonly used Assert-style is defaulted. See this: http://fluent.stirno.com/docs/#assert-intro

I too think that this was a strange design decision, but I'm sure that they had good reasons for going down the Expect-route. They have listen to many suggestions of supporting Assert as well and it's not in the tool. Good work OSS-developers!
Tuesday, March 11, 2014 1:11:24 AM UTC
Ah crap, @Michael again, forgot this in the last note.

I've used SpecFlow and FluentAutomation together quite a lot. I like it a lot. Here's a blogpost I wrote awhile back: http://www.marcusoft.net/2012/05/specflow-page-objects-and.html and here http://www.marcusoft.net/2013/04/PushTheHowDown.html

Please notice that this is based on a version that didn't have the PageObject support in FluentAutomation so I had to write them myself.

Wednesday, March 12, 2014 4:54:35 PM UTC
@Matt Honeycutt Aren't you testing just the Controller? And why should you? If there is too much logic in the controller your code smells. The whole reason for testing using eg. Selenium is too see whether the HTML/Javascript is working correctly in combination with the rest of the stack. Basically an end 2 end test.

What I find interesting is the way you spin up IIS Express to be able to test a website. Are you also using this in a TeamCity Continuous Integration/Delivery scenario.
Saber Karmous
Monday, March 17, 2014 3:48:26 AM UTC
Another pageobject approach:
https://github.com/leblancmeneses/RobustHaven.IntegrationTests

Why did I build the above? - answer on stackoverflow: http://stackoverflow.com/questions/20860148/looking-for-a-open-source-web-testing-automation-framework/20871839#20871839

There is selenium, kendo, and angularjs extensions that can be used independently.

As seen on nuget: http://www.nuget.org/profiles/leblancmeneses/



Tuesday, March 18, 2014 3:22:32 AM UTC
In this space, there's also a very neat looking F# lib entitled Canopy http://lefthandedgoat.github.io/canopy/
Friday, March 21, 2014 4:15:40 PM UTC
Chocolatey as in Chocolatey Nougat(Nuget). Makes more sense now right ;)
Michael Brown
Friday, April 11, 2014 2:47:17 AM UTC
:) i like this
Friday, April 11, 2014 2:50:39 AM UTC
Looks awesome. It's actually very similar to Nightwatch.js which I tried a couple of weeks ago. It's based on Node.js and Selenium server:
Wednesday, April 16, 2014 7:01:11 AM UTC
Hi Scott, nice post. Though i have used Microsoft's coded UI test (CUIT) in the past on a project. Fluent automation mentioned here does present an interesting choice of features and ease.
As you might be aware that very few test automation projects achieve desired success. In many organizations, Test Automation starts with lot of buzz and potential to reduce the test efforts drastically and optimize costs magically, but in reality more than often it ends up in spending unreasonably high amount of efforts in maintaining the developed automation suite not to mention requirement of highly skilled programmers (Java / C# / VB etc.) for developing automation scripts and the higher development time required to write automation scripts.

Following up on this, I came across and registered for a webinar on Simplifying Test Automation Using Automation Framework http://j.mp/1ivCoe4 which addresses above mentioned challenges, it looks a promising one.
sushant
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.