Scott Hanselman

T4 (Text Template Transformation Toolkit) Code Generation - Best Kept Visual Studio Secret

October 14, '08 Comments [30] Posted in ASP.NET | Programming | Tools
Sponsored By

Rob beat me to it. Blogging about T4 (the Text Template Transformation Toolkit) had been on my list literally for a year. He and I were singing its praises last night. Hopefully I can add something small to the conversation.

What's the story? Well, T4 is a code generator built right into Visual Studio. To be clear, you HAVE THIS NOW on your system…go play. Now's the time to introduce code generation to your company. If you're doing something twice or more, manually, in your company, generate it.

However, it's not deep-deep built in, because there's no item templates in File | New Item and there's no intellisense or syntax highlighting.

You don't need this, but if you want really get the most out of T4, first, head over to Clarius Consulting and get their "T4 Editor Community Edition." That'll get you some basic coloring. They have a pay version that gets you more if you want.

Now, go into Visual Studio and make a Console App (or any app) and add a Text File, but name it something with a .tt extension. You'll get a warning since it's a generator, that someone could generate evil. Click OK if you are cool with potential evil. ;)

Security Warning

imageNow, look in Solution Explorer at the .tt file. If you're using C#, you'll have a sub .cs file, or if you're using VB, a sub .vb file. That's the file that will hold the result of the generation. This is the same visual metaphor used to the express the template/generated file relationship with .designer files you've seen elsewhere in Visual Studio.

If you look in the Properties for the .tt file, you'll see it's using a CustomTool listed as the "TextTemplatingFileGenerator." This custom tool extensibility point is how things like XSD and WSDL code generators hook in to generate their artifacts.

T4 can generate any kind of text file, not just code. Whatever you like. You can use it in your projects, as above, or you can call T4 from the command-line.

Aside from Kzu and the folks at Clarius, Rob notes that Oleg Sych has a great collection of T4 resources. He's got some great Tutorials:

Here's some of the links from Oleg's blog.

Oleg also has a CodePlex project called T4 Toolbox that is a library of T4 templates that get added to File | New Item.

Also, check out Damien Guard's T4 templates that are a wholesale replacement of code that LINQ to SQL generates. Here's an example, where I use Damien's T4 templates against the sample Chinook Database.

image

Notice that I've named the .tt file the same as the .dbml, so Damian's code can find it. I also continue to let original LINQ to SQL generate it's .designer.cs file, but make that file's Build Action "None" so it's not ever compiled. That effectively puts Damian's code in charge.

Here's a screenshot showing a bit of Damian's T4 template using the syntax highlighting from the Clairus T4 Visual Studio free download. If I'd pay them, I'd get intellisense and syntax highlighting inside the code blocks also. It looks like a lot like ASP.NET Web Forms, or Velocity, or any templating language really. The code blocks are where your logic is and outside the codeblocks is the template for whatever you want to generate. Notice how Damien gets input and sets output. You have full control, you can read files off the file system, from with your project, etc. He sets the output extension also. I like to use .g.cs or .g.vb, myself. In this example his generated file is Chinook.generated.cs.

I particularly like Damien's example because he's swapping out parts of LINQ to SQL that he didn't like (the generated code) while keeping the part he did (the general mode, the designer, the dbml file.) If you don't like something, fix it.

Plus, it all works in Visual Studio without installing anything.

image

If you're doing Code Gen, or thinking about it, check out T4 as it's a great place to start. Also, search my blog for "Code Generation" as I was livign and breathing it with CodeSmith for the many years I worked at Corillian. Have fun!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Tuesday, October 14, 2008 8:06:36 PM UTC
Huzzah for T4! I'm glad you listed out Oleg's links - the guy is the Willy Wonka of T4 goodness. I'm so stoked on T4 that I'm currently redoing all my SubSonic templates in T4 as I write this.
Tuesday, October 14, 2008 8:19:57 PM UTC
I stumbled upon this a week or two ago and can't believe they don't publicize it more.
Tuesday, October 14, 2008 8:26:43 PM UTC
Just a word of warning, one of the things that I tried to do with T4 is to create proxy stub classes of one of my libraries using reflection - while the actual template generation works like gangbusters, T4 will load your references into VS's AppDomain to compile the template.

Since there is no way to unload an Assembly from an AppDomain once you load it, the end result is that you have to restart VS every time you compile because VS will report the file is locked :( Once I realized this it was pretty obvious what was happening, but it's definitely a hidden gotcha!

The solution is to create your own AppDomain, dynamically load your source library into it, reflect over the contents, then unload the entire AppDomain - a kind of a pain and it might be good in the future if VS would split T4 into its own executable and just run it instead of trying to do it in-process.
Tuesday, October 14, 2008 8:30:53 PM UTC
Paul - Sure, in that instance, I'd just use the command line T4 and a pre-build step. Why go to all the AppDomain trouble?
Tuesday, October 14, 2008 9:14:54 PM UTC
@ Paul: note that what controls which AppDomain to use is the T4 Host and not the T4 Engine so you can easily write your own T4 Host (it's pretty simply really) and run your templates into a brand new AppDomain.
Tuesday, October 14, 2008 11:18:14 PM UTC
Hi Scott,

there is a mistake in Oleg's last name spelling. It is Sych not Sync
Agnie Barto
Tuesday, October 14, 2008 11:19:41 PM UTC
Odd.. I'm trying this in VS2k8 with a VB project and it does not create the sub .vb file.

Dave K.
Tuesday, October 14, 2008 11:55:46 PM UTC
I'm getting the same result as Dave K. on VS2003 in C# -- also, no "wizard warning" is popping. Can T4 have been disabled? Earlier I had Resharper installed and after uninstalling it it did not restore some settings that it had disabled (in favor of its own facilities) when installed.
Wednesday, October 15, 2008 12:24:13 AM UTC
VB people - Make sure that the template looks like:

<#@ template language="VBv3.5" hostspecific="True"#>
<#@ output extension=".generated.vb"

Wednesday, October 15, 2008 12:25:38 AM UTC
CyberHerb - It works in 2008 out of the box. It works in 2005 if you install an AddIn, I believe. You said you're using 2003? It isn't in VS2003. That's .NET 1.1.
Wednesday, October 15, 2008 2:04:31 AM UTC
Scott's got it - T4 postdates VS2003 and isn't available for it. To get it in VS2005, you need to install the DSL Tools Runtime Redistributable and it is there right in the box in all versions of Visual Studio 2008.

Paul - T4 already does run in a separate AppDomain, but it only recycles it occassionally in order to get some better performance. As Victor says, you can control this by writing a host, but Scott's quick and dirty solution is maybe cheaper.

For the default VS Host, the AppDomain is recycled either
a) if 15 of the templates classes are more than 15 minutes older than the last used template
b) if caching is switched off (http://blogs.msdn.com/garethj/archive/2006/06/05/t4caching.aspx) every 25 uses of T4
Wednesday, October 15, 2008 2:05:57 AM UTC
Oh and Paul, you could also just try copying the assembly into temp and using LoadFrom before you reflect on it. The recycling means that the resulting leak won't be too unpleasant.
Wednesday, October 15, 2008 9:27:15 AM UTC
@Paul & @Scott

Oleg discusses the locking issue in his post on generating decorator classes. He suggests an alternative is to use the introspection engine developed by the FXCop team rather than reflection.

In an article on FXCop he links to it says,

"The use of the Introspection engine allows for much faster analysis and supports multithreaded analysis. Unlike the Reflection engine from previous versions, in the Introspection engine the assemblies you're analyzing are not locked so you won't need to shut down FxCop to do a fix and recompile of those assemblies."
Wednesday, October 15, 2008 12:58:09 PM UTC
Doesn't seem to support express editions. Anyone know if that's correct?
AllanN
Wednesday, October 15, 2008 1:21:51 PM UTC
This is very nice - and I like the editor, too. But it's just a shame it doesn't integrate with the rest of the msbuild based project system...
Wednesday, October 15, 2008 2:32:08 PM UTC
For those who don't see the .vb or .cs file, be sure to click "show all files" in the solution explorer. It will show up then.

Jon
Wednesday, October 15, 2008 3:48:36 PM UTC
So just to add a bit of a comparison, here are a few of the advantages that CodeSmith provides over T4:

1. MSBuild integration so that your code generation can happen as part of the build process and while building from the command line or on your build server.
2. Ability to generate multiple files or single file. You can also control codebehind settings for each file generated.
3. Ability to easily specify metadata or properties to drive the templates and ability to setup multiple templates to run as a batch.
4. Ability to set the build action of generated files. ie. Maybe set a file to be an embedded resource.
5. Templates run in a separate process so that the referenced assemblies are not locked.
6. Ability to execute complex templates as snippet generators from the editor.
7. Source control integration.
8. Ability to automatically add necessary project references to the project.

Those are just a couple of the differentiators from T4 to CodeSmith's Visual Studio integration and doesn't even cover any of the benefits of the template syntax and flexibility that CodeSmith provides. Here is a link to a quick video that goes over the Visual Studio integration features of CodeSmith:

http://www.codesmithtools.com/video/visual-studio-integration.html

Eric J. Smith
Wednesday, October 15, 2008 6:15:32 PM UTC
@ Eric: I don't see that as a fair comparison, you're comparing a transformation engine (T4) vs. a tooling product + a transformation engine (codesmith).

Your list may sound like this could be a limitation of T4 technology while it isn't, all the points you list can be achieved using T4 and writing proper code. Moreover, most of them are already out there in form of open source projects like the Web Service Software Factory from p&p which generates multiple outputs from a single template.

On the tooling side AFAIK there is no T4-based product alternative to compare to right now. If you're into T4 there are a few standalone lifesavers as of today like our Clarius T4 Editor which -by the way- IMHO provides a much better IntelliSense experience than CodeSmith as it reuses VS existing IntelliSense support for this instead of re-inventing its own.

And on the engine side, while I'm not familiar with CodeSmith as to tell which one of the two engines is better, I've been using T4 engine extensively and I'm very happy with it. A major point to note is that you can redistribute for free the T4 Engine to VS2005 users and you don't need to distribute at all for VS2008 as it is already built-in while this isn't true for CodeSmith right? Last time I checked one had to pay a redist for this thus I'm may be wrong...
Wednesday, October 15, 2008 10:18:22 PM UTC
The other side of the medallion is that T4 is far from being a mature technology. Its usage shouldn't be encouraged with its current situation :

- To achieve even simple things you need to write your own host and directory processor. Paul's problem is one of them that this approach solves, but there are tons of things like that and it requires a lot of codelifting (needless to say that nearly impossible to unit test).
- Visual studio experience isn't the best. With 2008, you have some support(if you think running generation engine in each ctrl+s is indeed a support), with 2005 you are totally out of the game. Other than paying to clarius consulting, who provides a very basic intellisense and higlighting and sends the bill to your house, you have no other option.
- The errors are *really* cryptic. There is no way that you can find which line causes the error without looking at the temporary file, which is most of the time is not even there(cleaned up because of the exception). The messages are obscure (Got an error in Line : 0 ? go figure yourself now).
- Very wierd design decisions, such as relative paths are not relative to template itsself's path but to app's path.
- Documentation quality in MSDN is nowhere near other VS technologies documentations. Very trivial examples.
- In custom hosts you can only communicate with the engine by magic strings and on the fly code generation.
- Templates are hard to test, debugging is a nightmare.

And tons of other things.

The product has a long way to go to be even close to its competitors.

Also, just because that we talk about T4 shouldn't mean that we have to promote code generation. So I'd amend following sentence :

>> If you're doing something twice or more, manually, in your company, generate it.

to my own view

>> If you're doing something twice or more, manually, in your company, refactor it.
Wednesday, October 15, 2008 11:44:01 PM UTC
@Sidar,

Again, you seem to be comparing a complete a full-blown CodeGen product like CodeSmith against the T4 engine which, just in case it's not clear enough, is not a full-blown CodeGen product and as such offers minimal tooling support. This is not really big news isn't it?

My point is that comparing oranges to apples doesn't make much sense -- I also think a company could build upon the current T4 engine and offer a competitive product to any of the CodeGen tools out there, and it is against such a tool what you should compare.

As I worked on the T4 Editor I'm particularly interested in why do you believe the IntelliSense and "highlighting" (I'm guessing this is syntax coloring) of T4 parts + C# code is "very basic" as I believe it's pretty complete offering an experience almost in a par with the ASP.Net editor. Which other tool offers a better experience in this regard?

And yes, we do charge for our Professional Edition (note we also offer a free Community one) as we're part of the weird species that like to get paid for its work :-)





Thursday, October 16, 2008 12:47:31 AM UTC
@Victor

>> Again, you seem to be comparing a complete a full-blown CodeGen product like CodeSmith against the T4 engine which, just in case it's not clear enough, is not a full-blown CodeGen product and as such offers minimal tooling support. This is not really big news isn't it?

I didn't even have CodeSmith in mind while writing this, and didn't mention it once. I won't compare an apple to an apple tree. Most of the problems I listed are not even tool related problems, they are architectural or design issues and related to T4 Engine itsself. These needs are basic ones, I didn't pick them up from any product - they are pains from real life.

However, to play the game fairly people need to know what expects them in the wild forest. They need to know what they are gonna pay for, and what they are not going to get. T4 is definitely an orphaned child of VS product unit line, and comes with the least-if no is rude-support OOTB.

On Clarius T4 Editor:

First I have to thank and say that as it is the first and only work in the arena, this editor eased my life a lot. Though, it is still far from carrying user experience to a next level.

- Intellisense is poor : Do you have intellisense in c# code written in templates ? Do you have autocomplete in the imported assemblies ? Do you have intellisense for included templates ? Or Custom Directive Processors ? Do you have intellisense comments that shows what is used for what ?

- Syntax Coloring is poor: Only directives have appropriate colors, C# codes inside (<#+ +> that is) is not colored. Actually, other than the directives at the top, the template is barely correctly colored.

>> Which other tool offers a better experience in this regard?

There aren't, and that's indeed the main problem. Even with your extra money, you still can't buy a normal visual studio experiment.

>> And yes, we do charge for our Professional Edition (note we also offer a free Community one) as we're part of the weird species that like to get paid for its work :-)

Fair enough, so there is no problem in getting people know this before jumping into T4 generation world, is there ?
Thursday, October 16, 2008 2:29:53 AM UTC
@Sidar,

Sorry if I wrongly interpreted the first part of your last response, I thought you were -like Eric- comparing the T4 engine against a full-blown codegen product.

Note that I do agree with you on the fact that there are some existing paint points today with T4 engine that you need to workaround and that -as tooling is mostly non-existent- pretty much everything requires coding. I do agree too that parts of it could benefit from some re-design, no doubt, and I hope future T4 versions will improve all these. But at the same time I don't consider these as an adoption showstopper. There are several projects out there that have succesfully used T4 and this technology is fully backed by Microsoft and officially supported through PSS. All this is enough for me to go with it for most cases. I believe this is the point were we disagree and it's ok.

Now regarding the Clarius T4 Editor, it looks like you've not tried our Professional Edition, because:

>>>Do you have intellisense in c# code written in templates?
>>>Do you have autocomplete in the imported assemblies?
>>>Do you have intellisense for included templates?
>>>Or Custom Directive Processors?
>>>Do you have intellisense comments that shows what is used for what?

ALL of the above is supported by our Pro Edition :-) That's why I was surprised to hear you found the IntelliSense support was "poor" as I sincerely think -bias apart now!- is pretty good.

Maybe you were using our old beta or the Community Edition? These two don't support much of the above.
Thursday, October 16, 2008 3:53:18 PM UTC
Awesome Scott,

Thanks for enlightening me about this tool inside VS2008.

Thursday, October 16, 2008 4:29:31 PM UTC
Anyone got an example of how to set up "use the command line T4 and a pre-build step" ?
Awe
Thursday, October 16, 2008 7:46:47 PM UTC
I have been using the Clarius T4 editor (pro edition) for a few weeks now, and had the beta before that, it makes a massive difference to the T4 IDE experience. Victor is right that the features mentioned are supported and for me, working with DSLs and T4 a lot, the inclusion of VS native language service means I can use extension methods and linq while getting full intellisense! It has even helped us refactor our templates from the mess we had before simply because of the improved ide support and the fact that discoverability reduces duplication in a team environment.

As for error line numbers and relative paths for includes, not really sure where you're coming from on this one. Neither of those are a problem, also with Clarius' editor you can see the generated cs file at any time. (I don't work for Clarius)

There are some things which are pretty annoying in T4 like debugging support but, to be honest, if you get to the point that this is a problem, you need to refactor, massive templates really are hard to work with, but do you write code like that?

Building custom hosts is sometimes necessary, especially if you want build script support but really this is a piece of cake and fairly well documented.

Pete
Friday, October 17, 2008 11:27:58 AM UTC
Hi Scott (or anyone who can answer the following question),

What is a common usage for such a technology? I would generate ORM stuff with it because we already have so many ORM software including LINQ to SQL with that. Is there any cool templates you guys developed and that you could reuse?

I just need some concrete exemple of using such technology.

Thanks!
Saturday, October 18, 2008 1:12:14 AM UTC
Maxim, a great example of the use of T4 is the DSL Tools which are part of the Visual Studio 2008 SDK. They use T4 templates to generate all the code for a graphical designer for a DSL from a DSL describing the domain model. And, of course, most DSLs generated through the Toolkit will use T4 to generate code or other text-based artifacts. The Configuration Section Designer (http://www.codeplex.com/csd/) generates strongly-typed configuration section classes and XML schema from a model of the configuration section desired.

Another is the Service Factory (http://www.codeplex.com/servicefactory/) which uses T4 templates to generate code from three DSLs into several projects.
John Saunders
Wednesday, November 05, 2008 1:45:39 PM UTC
Why cant i use .tt file that are automatically built in Wix Projects?
Friday, November 14, 2008 6:41:18 PM UTC
Nice to hear from this...

But my question its this:

Does the T4 have an api so i can do code generation at run time? I think any tool should always have the following features:

- API
- Command line

With dynamic application and build tools, continuous integration these two features are essentials. Of course that having an API we can create the command line utility. But withou API i think its a waste of time. Working only in Visual Studio just make it more "license oriented", this means, more money to MS.

Thanks :D
Rafael
Friday, November 14, 2008 10:25:30 PM UTC
Yesterday I tried adding a T4 template to a Smart Device project, but it doesn't work because the Compact Framework doesn't include the System.CodeDom namespace. But I found a way to make it work if you or your readers are interested:

T4 Templates for the .NET Compact Framework

--Jeff
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.