Scott Hanselman

The Weekly Source Code 58 - Generating (Database) Test Data with AutoPoco and Entity Framework Code First

January 5, '11 Comments [26] Posted in Open Source | Source Code
Sponsored By

I was messing around with Entity Framework Code First for data access recently (what I like to call EF Magic Unicorn) and had a need to create a bunch of test data. Such a chore. It's totally no fun and I always end up slapping the keyboard thinking that someone else should be slapping the keyboard. Test data for 10 instances of a class is easy, but 1000 is some how less inspiring.

Sure, there's lots of software I could buy to solve a problem like that, but meh. I dug around some and found an open source framework called AutoPoco from Rob Ashton (@robashton). First, awesome name. I like using open source projects with cool names. It's kind of like reading a book with a great cover. Makes you feel good jumping in. Then, I started working with it and realized there's some substance to this modest little library. All the better, right?

AutoPoco's CodePlex project says:

AutoPoco is a highly configurable framework for the purpose of fluently building readable test data from Plain Old CLR Objects

Here's the general idea. You need 100 of something (or 5, or whatever) setup a certain way, so you ask AutoPoco to construct a bunch of you, and it's so. It's that easy.

For example, please get me 1000 objects of type SimpleUser and make their FirstName's all "Bob.":

IList<SimpleUser> users = session.List<SimpleUser>(1000).Impose(x => x.FirstName, "Bob").Get();

Here's where it really shines, though:

session.List<SimpleUser>(100)
.First(50)
.Impose(x => x.FirstName, "Rob")
.Impose(x => x.LastName, "Ashton")
.Next(50)
.Impose(x => x.FirstName, "Luke")
.Impose(x => x.LastName, "Smith")
.All().Random(25)
.Impose(x => x.Role,roleOne)
.Next(25)
.Impose(x => x.Role,roleTwo)
.Next(50)
.Impose(x => x.Role, roleThree)
.All()
.Invoke(x => x.SetPassword("Password1"))
.Get();

This says:

Create 100 users
The first 50 of those users will be called Rob Ashton
The last 50 of those users will be called Luke Smith
25 Random users will have RoleOne
A different 25 random users will have RoleTwo
And the other 50 users will have RoleThree
And set the password on every single user to Password1

Effectively, the sky's the limit. You can also give AutoPoco more advanced requirements like 'make emails meet these requirements," or "call this method when it's time to get a password." The idea being not just to make test data, but to make somewhat meaningful test data.

This got me thinking that since we're using POCOs (Plain Ol' CLR Objects) that I could use this not only for Unit Tests but also Integration Tests and Smoke Tests. I could use this to generate test data in a database. All the better to use this with the new Entity Framework stuff that also uses POCOs.

For example:

public void MakeTestData()
{
IGenerationSessionFactory factory = AutoPocoContainer.Configure(x =>
{
x.Conventions(c => { c.UseDefaultConventions(); });
x.AddFromAssemblyContainingType<SimpleUser>();
});

IGenerationSession session = factory.CreateSession();

IList<SimpleUser> users = session.List<SimpleUser>(1000)
.First(500)
.Impose(x => x.FirstName, "Bob")
.Next(500)
.Impose(x => x.FirstName, "Alice")
.All()
.Impose(x => x.LastName, "Hanselman")
.Random(250)
.Impose(x => x.LastName, "Blue")
.All().Random(400)
.Impose(x => x.LastName, "Red")
.All()
.Get();

SimpleUserDatabase db = new SimpleUserDatabase();
foreach (SimpleUser s in users)
{
db.Users.Add(s);
}
db.SaveChanges();
}

And boom, I've got 1000 users in my little database.

I've talked to the author, Rob, and I think that the Session creation and Factory stuff could be made smaller, and the loop at the bottom could be reduced a line or two. Rob's a practical guy, and I look forward to where AutoPoco goes next! All in all, what a useful library. I can see myself using this a lot.

You can get AutoPoco from CodePlex, or even better, from inside Visual Studio using NuGet (http://nuget.org) via "install-package autopoco."

Enjoy! This library has 36 downloads, but deserves a few orders of magnitudes more. I'll do my best to showcase more open source libraries that deserve attention (and more importantly, to be used!) going forward. Feel free to email me suggestions of insanely useful libraries.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Wednesday, January 05, 2011 9:33:59 AM UTC
Thanks a lot scott, sounds very cool! I was actually looking for something like that for use when designing my WPF of Silverlight applications. It's always nice to have some data to work with :)
Wednesday, January 05, 2011 9:47:07 AM UTC
Look like a nice library, but i still have some big problems with mock.
How can I mock process like send a mail, and what should be a good way to mock tests that should check if the database configuration is alright and the object was persisted correctly.
Wednesday, January 05, 2011 10:03:09 AM UTC
There is similar lib: http://nbuilder.org/, but as far as i know it isn't avaliable through nuget. But you could take a look on it :)
Wednesday, January 05, 2011 10:04:24 AM UTC
There's few more frameworks for this, I've used NBuilder several times, it is open source too, but I'm doubting if such libraries are suitable for casual unit testing.
Wednesday, January 05, 2011 10:09:12 AM UTC
Nice! NBuilder looks very cool. I'll check it out, although Example #2 on their site (the date stuff) was pretty intense! ;)
Wednesday, January 05, 2011 10:14:02 AM UTC
This should not be necessary:

x.Conventions(c => { c.UseDefaultConventions(); });

Mike
Wednesday, January 05, 2011 10:28:16 AM UTC
AForge.Net by Andrew Kirilov. http://www.aforgenet.com/ Nice image manipulation. It does everything and everything fast ;-).
Wednesday, January 05, 2011 12:26:01 PM UTC
I would also second NBuilder, as I have been using it for a couple of years now and have been very pleased with it. Hopefully they will make it available via NuGet.
Wednesday, January 05, 2011 12:57:56 PM UTC
Nice libraries! Thanks!
Is there a library / T4 template for reverse engineering POCO's from database?
Puneet
Wednesday, January 05, 2011 1:20:42 PM UTC
I think the link to the Magic Unicorn Article is borked.
Matt
Wednesday, January 05, 2011 1:47:51 PM UTC
I dont like this.

Tests with random data are bad. If you run a test and it fails, what do you do? The next time you run it, it succeeds. It's not reproducible, you may as well have your users find the bugs.

Discuss.
El Guapo
Wednesday, January 05, 2011 1:57:10 PM UTC
AutoPoco is pseudo random, all datasources will create the same data per session (ie, per test)
Wednesday, January 05, 2011 3:00:12 PM UTC
There's one more framework like that - http://autofixture.codeplex.com/
Wednesday, January 05, 2011 3:34:35 PM UTC
+1 for NBuilder, it's fluent syntax is easier to read IMHO. We use it all the time over at Russell.
Wednesday, January 05, 2011 4:58:01 PM UTC
Have a look at NCalc.codeplex.com - opensource mathematical expression evaluator with some neat tricks
Wednesday, January 05, 2011 11:23:51 PM UTC
Another one here that has been using NBuilder for a while :)

I was trying to use NBuilder as an In-Memory Persistence / DB ... but could get past the problem of retrieving complex heirachies of data. eg. Parent has 3 children ... but sometimes we want to just grab the children (as if they are parents/root aggregates). <-- i hope i use the right lingo, there.

Would love to see some detailed examples of using NBuilder/AutoPoco for Unit Tests/Intergration Tests and even Smoke Testing. *hint hint smart people* :)
Thursday, January 06, 2011 9:00:21 AM UTC
You should check out AutoFixture as well.
Friday, January 07, 2011 9:00:16 AM UTC
Seems pretty interesting! And by the way, you advertising this lib will certainly have its download stats increase quickly (it already started: http://autopoco.codeplex.com/stats :))
Friday, January 07, 2011 9:55:19 PM UTC
Somewhat related and in a desperate attempt to increase usage :) I maintain FsCheck, a random testing framework. It is written in F# but has a C# friendly API as well (unfortunately the latter is undocumented at this point).

It has an embedded DSL for building random generators of values for your type, and another for specifying the actual test. Neat feature: if your test fails for a particular example, it is "shrunk" to become the minimal example that fails the test.

Here's something like your first example in FsCheck - first the generator:

var anyPerson = from firstName in Any.ValueIn("Rob", "Luke")
from lastName in Any.ValueIn("Ashton", "Smith")
select new Person() { FirstName=firstName, LastName=lastName, Password="Secret" };
And the test:
Spec.For(anyPerson, person => person.FirstName != null)
.QuickCheck("FirstName should not be null");

which when you run it gives:
FirstName should not be null-Ok, passed 100 tests.
Tuesday, January 11, 2011 5:15:43 AM UTC
Hi Scott,

Have a reasonable understanding of the AutoPoco framework now. I am currently struggling on two things. Maybe u know the answers straight away -

1. If I configure creation of Object A and also configure creation of a collection of child objects within A i.e. IEnumerable. Thats fine and I am
able to do that by writing a custom DataSource implementation for B. But, If I have a property on B that identifies its parent i.e. Object A, the AutoPoco configuration creates a new instance of A within B (based on the configurations of A I assume). How do I stop the framework from doing that and get a handle on the calling parent object instance A, so that I can then set the Parent object property on B correctly. For instance, Is there a way to access the calling object in the Next() function through the Session?

2. Is there curently a datasource that generates an ID for the objects? I couldnt find one - so I am assuming I will have to write a IdDatasource with a
dictionary of available Ids for a particular object type?

Thanks heaps

Regards

Jatin
Jatin Dua
Tuesday, January 11, 2011 1:31:30 PM UTC
Hi, Scott ! In Ruby community they have a gem named 'Faker' which will authentically fake your data not just creating 50 users Rob Schneider. And also Faker knows what is a authentically username, an email address, a name and a last name.
Also I saw that someone did something similar in .NET on codeplex named 'Faker' (http://faker.codeplex.com/) although I didn't quite saw if it is similar or not.
Tuesday, January 11, 2011 7:21:35 PM UTC
I built Objecthydrator http://objecthydrator.codeplex.com...it will generate fairly real looking data. Though in many respects not as mature as some of the others...but I think it does names,addresses, email and company names pretty well.
Ryan Smith
Wednesday, January 12, 2011 11:18:44 PM UTC
Jatin Dua ::

1) Setup(x=>x.Property).UseParent(); is what you are looking for (in trunk)
2) Yeah, IntegerIdSource


Any other questions can be asked in the forums at http://autopoco.codeplex.com
Wednesday, January 12, 2011 11:53:58 PM UTC
Thanks heaps Rob
Jatin Dua
Wednesday, January 26, 2011 5:11:00 AM UTC
NBuilder is indeed a great project. I started using it fairly early on and ended up contributing a small amount of code to the project. So I went ahead and created a NuGet package for NBuilder and submitted it to the gallery, so it's now available to install via NuGet.

Scott, as for the "intense" date stuff, that's of course totally optional. :-) You can just set dates like you normally would (using new DateTime(...) or DateTime.Parse(...) or whatever). I believe the fluent syntax was an experiment by Gareth (the author of NBuilder) to see how far he could push a fluent API. It can be nice in certain situations and for occasional use, but probably not something you'd want to use if you're dealing with a lot of dates.
Friday, January 28, 2011 9:58:38 PM UTC
Hi Scott

The dates in NBuilder can be as simple as just:

July.The1st

(uses the current year)

That particular example is just the full works!

Gareth (founder of NBuilder)
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.