Scott Hanselman

Tiny Happy Features #3 - Publishing Improvements, chained Config Transforms and Deploying ASP.NET Apps from the Command Line

August 12, '12 Comments [39] Posted in ASP.NET | ASP.NET MVC | Tiny Happy Features
Sponsored By

(UPDATE: See other Tiny Happy Features)

Publish Profiles are stored with the projectAt some point soon lots of people are going to start writing these epic blog posts about Visual Studio 2012. They will include LOTS of screenshots (some good and some bad), some small code samples and minimal context. I can't speak for other teams; I can only talk about what we worked on. The <AngleBrackets/> folks in Azure Platform and Tools (ASP.NET, IIS, WCF, EF, Azure much and more) have been putting a lot of work into what I sometimes call "Death by a Thousand Tiny Cuts." It's the little irritants that are as frustrating (or more so) as the big missing features.

Rather than a giant super post (although I'll do that at some point) I thought I'd showcase some Tiny Happy Features that the team worked on just because it made life better. Some are large some are small, but all are tiny happy features.

Publishing and Deployment Improvements

Visual Studio 2012 has a number of subtle but significant improvements to publishing. Of course you likely know you can push code to Azure or AppHarbor using Git. You can publish your web app with Web Deploy which I demonstrated at Mix in my talk Web Deployment Made Awesome: If You're Using XCopy, You're Doing It Wrong. You can see a quick closed-captioned video I did of Web Deployment to Azure in just 4 minutes over here.

Visual Studio added web.config transforms a while back so you can have a Web.config along with a Web.debug.config with the changes you want for debug time, and web.release.config for release time. You can add the free SlowCheetah add-on and get transform support for all your config files, not just web.config.

However, things fell down because Web Deploy and Visual Studio didn't have a way to easily represent Dev, Testing, Staging, Production, Whatever. You could have build configurations but they didn't relate to deployments, which doesn't reflect a developer's reality.

Publishing with Environment-specific Configurations

Publish Profiles are either created manually, or now in VS2012 downloaded as .publishsettings from your host and imported into Visual Studio. In Visual Studio 2012 publish profiles are stored along with your code in the Properties folder so they can be checked in and used by others.

Visual Studio 2012 adds deployment config transforms

You can rename a profile whatever you like by changing the name. Here I've named my publish profile "Production."  There's the coolest part.

PublishProfiles live in the Properties folder

I can make a Web.config file with the same name as my Publish Profile and that transform will be run after the build transform.

Even better you can right click on a Transform now and select Preview Transform and see not only the results, but a Diff of the results.

Transformed Web.config ( transforms applied: Web.Release.config, Web.Production.config)

See right there, and in the large image above? It says "Transformed Web.config ( transforms applied: Web.Release.config, Web.Production.config)." Build transforms happen first, then Publish Transforms. Right now you need to create the file with the same name yourself but we'll be adding tooling for this.

This means I can do things in Web.debug.config like changing compilation options while Web.production.config is for changing connection strings, setting log levels, and adding specific Production settings.

Deploying from the Command Line

Deploying your web application from the command line has long been possible but it's been a pretty obscure and frustrating affair. Now that the Publish Profiles can live with the project and Web.config transforms can be chained in you can publish from the command line much easier.

msbuild MySolution.sln /p:DeployOnBuild=true;PublishProfile=Production;Password=poo

If you have a untrusted certificate on the deployment server you haven't added to your local certs you'll need to add AllowUntrustedCertificate=true as an acknowledgement.

I just make a Deploy.bat that looks like this with a %1 for the Profile Name.

msbuild MySolution.sln /p:DeployOnBuild=true;PublishProfile=%1;AllowUntrustedCertificate=true;Password=poo

Then I can just do

Deploy Production

or

Deploy Staging

I've found these outwardly small but impactful changes have made it a lot easier for me to deploy sites. Deployment shouldn't be hard. I stand by my original statement: If you are using XCopy (or Windows Explorer or FileZilla) to deploy your website, you're doing it wrong.

If you are using Web Sites rather than Web Products, head over to the Web Team blog (and subscribe, too) and add your voice to the comments around our teams' plans regarding Website projects and Web Deployment Projects.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Sunday, August 12, 2012 10:17:39 PM UTC
These are nice improvements, but I would really like a nice way to do config transforms without using webdeploy, since we have to build in non-VS projects and we're using LZMA compression for our packages, so the weirdly structured webdeploy packages just make things more complicated. At the moment we're just using Hg patch queues to push on a transform before building a package, which is pretty broken, but I'm looking at using hacky-looking ways using internal MSBuild targets or 3rd party reimplementations of config transforms.
Sunday, August 12, 2012 10:51:59 PM UTC
I like that the deployment is being invested in, even if it's just a small team. The main problem at the moment of one of documentation, particularly of the MSBuild stack on top of MSDeploy.

If you get the chance, could you ask someone on the Publishing Pipeline team to take a look at the following Stack Overflow question? Basically, I'd like to "publish" to a package and later "publish" that package to another profile (without rebuilding it).

http://stackoverflow.com/questions/11683243/deploying-an-existing-package-using-publish-profiles
Sunday, August 12, 2012 10:52:58 PM UTC
Sorry, I noticed the support for links after I posted

Here is a direct link to the SO question
Monday, August 13, 2012 1:08:47 AM UTC
Seeing what the transform changes is nice, however i'd love a way to see what it will do to the production web.config when it is deployed. Because using the IIS admin interface can modify the web.config and some changes don't always go through the right channels it can mean you accidentally overwrite something important.
Betty
Monday, August 13, 2012 3:34:37 AM UTC
@Simon, you don't need Web Deploy to do XDT XML transforms. In fact Web Deploy itself does not know how to execute a transform. For web projects the web.config is transformed using MSBuild during the publish/package process. We have created an XML task, TransformXml, to execute the transforms. You can reuse this task if you want. I have blogged showing how you can do this at Config transformations outside of web app builds. There are other posts on TransformXml on my blog which you also may be interested in.
Monday, August 13, 2012 3:50:07 AM UTC
@Richard I realize the documentation is an issue, especially for the cases which you describe. I am working to better this situation.

I looked at your question and it made me realize that accomplishing what you want is more difficult than what I thought. One of the tricky bits is to construct the full MSDeploy service URL. But I have created an add in which I think you may find useful. Check out my 5 minute video. I could update that to support .pubxml/.publishSettings files. Let me know what you think. I could also try and put a solution together strictly in MSBuild. Let me know what you prefer.
Monday, August 13, 2012 3:57:23 AM UTC
@Betty I'm not sure if I understand your reply can you clarify what you are saying?

FYI Our VS publish experience is "out-of-band", meaning that we can update it later easily, so we will be shipping updates on a regular basis. One feature that we are considering adding is to let you see the file on your PC and the file on the server which is getting updates in a diff viewer. You would get this functionality on the publish dialog, during the file preview. For web.config it will show you the diff of the web.config on your local PC after MSDeploy updates the values with parameters and the web.config on the server. Would this meet your needs?
Monday, August 13, 2012 5:54:17 AM UTC
@Sayed - PackageWeb looks interest, I'll take a look. I probably would prefer a non-powershell solution, though.

I don't actually mind holding onto the pubxml. The main issue is that there's not a solid enough separation of "package" and "deploy" in System.Web.Publishing.targets

Specifically, the "MSDeployPublish" target can't simply write out a SetParameters and use the package as a source, it has to also _define_ the parameters (which causes other problems).

I'm happy to continue this chat on the Stack Overflow question, as that site supports a chat mode.
Monday, August 13, 2012 5:54:25 AM UTC
*looks interesting
Monday, August 13, 2012 8:05:50 AM UTC
Please tell me you added config transforms for all .config files and not just web.config.

We have to use a few "hacks" in our project files now to transform files like conenctionstrings.config and whatnot just cause of this silly hard-coded limitation of only transforming the file named 'web.config'
Monday, August 13, 2012 8:41:35 AM UTC
I second Erik: I also want to transform any and all .config files easily WIT NO VSIX.
Mike
Monday, August 13, 2012 11:13:35 AM UTC
@Erik / @Mike - I'd recommend using transforms for debug vs release stuff and using msdeploy's parameters for the per-environment stuff. As of v3, msdeploy's pretty much has feature parity with transforms.
Monday, August 13, 2012 11:26:25 AM UTC
debug vs release is usually 1 change form true to false (<compilation debug=true/>), per environment changes are quite a lot.

This limitation to only web.config makes no sense at all since the transforms only happen when you publish/deploy, not when you just build.

It's a great feature that falls short due to a pointless file name limitation.

Monday, August 13, 2012 2:53:19 PM UTC
Thank you for the write up!

Dumb question: how is this practically any different from having one build configuration per deploy environment? If my deploy is "msbuild my.sln ..." as above, then deployment and build are the same step in my workflow.

What I would really like is a build step that creates a package "tarball" that is either debug or release bits. Then to deploy it to staging, I take that tarball and apply a config transformation for the staging environment. Then when QA wants to, they can take a tarball that has been approved on staging and deploy it to production, *with no changes or recompilation* other than applying the production config transformation. They should be able to do this with 100% certainty that the ONLY difference between staging and release has been that final config transformation.

If instead build and deploy are one step as above, then every deployment package is a unique output of the *compiler*, not just of config transformation.

I feel that I must be missing something really basic. :-)
Monday, August 13, 2012 4:39:48 PM UTC
Having used config transforms and build configurations to manage environments, this looks amazing!


1. Is there a way to have devops manage production credentials without them being in the solution or at least not be accessible by devs?
2. Is there a way to use this in VS2010?

Monday, August 13, 2012 6:04:01 PM UTC
Transformations are nice, but the overtime changes in web.config are minimal in my experience to go over the pain of doing it. I just ignore web.config when publishing.


Do you know if there is a way to make publish wizard to copy App_offline.htm while it is publishing the site?
Eduardo
Monday, August 13, 2012 6:08:48 PM UTC
So cool tiny features added in VS2012 :)

But let me to describe a related publishing issue which We've faced with it in our recent project. When I start to publish our MVC App using VS through FTP, It takes very very long time to complete and most of the time I'll regret it and try XCopy instead :( It should be much faster! I don't know what's the problem.
Monday, August 13, 2012 7:49:23 PM UTC
Scott, is there any hope that you'll have a Tiny Happy Feature sometime in the future that covers an easier way to manage switching between a set of startup projects and a single startup project? (Think scenarios where I want to run gui front ends but not the server and vice versa.)
Shawn
Monday, August 13, 2012 10:48:07 PM UTC
@Mohammad - FTP is a chatty protocol so deploying to it is probably going to be slow compared to XCopy.
Tuesday, August 14, 2012 12:31:28 AM UTC
@Kareem Sultan we are making all these features available for VS 2010 on 8/15. After 8/15 you can get the bits at https://www.windowsazure.com/en-us/develop/net/#, click on the big Install button and then select VS 2010.
Tuesday, August 14, 2012 12:40:47 AM UTC
@Eduardo I have responded back to your question. I've blogged about how you can take your app offline at http://sedodream.com/2012/01/08/HowToTakeYourWebAppOfflineDuringPublishing.aspx
Tuesday, August 14, 2012 12:45:07 AM UTC
@Brad the difference is that you do not have to create a build configuration for a concept which is publish related.

FYI you should look at my 5 minute video at http://sedodream.com/2012/03/14/PackageWebUpdatedAndVideoBelow.aspx
Tuesday, August 14, 2012 1:21:21 AM UTC
@Sayed - I've created an MSBuild-based package-to-publish-profile deployment script.

The main issues with using the standard targets for this are outlined in the script's comments, but are:

UseDeclareParametersXMLInMsDeploy=true doesn't work because MSDeployPublish->ExportParametersFile uses DeclareSetParameterFile which causes the arguments to be _declared_ (not set). Changing this to SetParameterFile fixes it, but only makes sense when publishing from a package.

UseDeclareParametersXMLInMsDeploy=false doesn't work because MSDeployPublish->VSMSDeploy and MSDeployPublish->MSdeploy both use SetParameterItems for inlined parameters, which doesn't seem to apply them. Changing this to SimpleSetParameterItems fixes the issue (which may also not be relevant to non-package based publishing).

Let me know if either of these issues are fixed in the 15/08 release and I'll update my script accordingly.
Tuesday, August 14, 2012 1:52:22 AM UTC
@Mike/Erik the built in support has not changed. I tried my hardest (and trust me when I say that) to get this integrated into the other projects. At the end of the day the team which would have to do the work was already over burdened and could not absorb the additional work.

Fortunately you can get this support from my add in at SlowCheetah. I realize its not ideal (because it's an add-in) but that is where we are at.

Sayed Ibrahim Hashimi
Thursday, August 16, 2012 7:35:00 AM UTC
@Sayed thanks for the addon tip, but we are for now gonna stay with the "hack" in the web project file so that our buildserver (teamcity using msbuild) also can transform config files.
Wednesday, August 22, 2012 7:38:40 AM UTC
Appharbor looks very cool.
But is there any way that we could publish our websites/apps to our own servers in a "AppHarbor-stylish way" ?
I mean so that we see if an application is working correcly and can revert to a previous version if we need to?
Mattias
Thursday, August 23, 2012 5:05:10 AM UTC
Mattias - Sure, use Project Kudu (on Github) or use Azure.
Friday, August 24, 2012 10:37:49 PM UTC
I'm very satisfied that MS at last is recogninzing that deploy environments is orthogonal to compiler optimizations.

But I don't see that you solved how anyone can save credentials for production environment in TFS yet, without writing custom scripts or making a mess in TFS.
devops
Friday, November 02, 2012 10:31:12 AM UTC
I've been using the environment specific configurations in VS2012 and they rock. However I sadly realized that they don't work when publishing to Windows Azure from within Visual Studio.

I'm trying to deploy a Web Role to Azure and the only way to have the web.config being transformed as I want is to go to the project and define a build configuration for each different environment, which is exactly what I did before VS2012 anyway. And all that just to have the web.config transformed for each different environment.

Is there a reason the Windows Azure Tools don't support this otherwise great feature?
Kostas
Thursday, January 17, 2013 8:25:58 AM UTC
@Brad Williams, what you are looking for is PackageWeb. It allows you to create a single web deploy package with transforms embedded within it. Then you can use the generated .ps1 file to publish including transforming.

Checkout my 5 minute video. http://sedodream.com/2012/03/14/PackageWebUpdatedAndVideoBelow.aspx

If you are interested in this being included in VS please let us know!!! You can vote at http://aspnet.uservoice.com/forums/41199-general-asp-net/suggestions/3474938-add-the-ability-to-transform-web-config-without-re
Thursday, January 17, 2013 8:31:37 AM UTC
@Kareem Sultan


"1. Is there a way to have devops manage production credentials without them being in the solution or at least not be accessible by devs?"


We do not have a good story for this today.


2. Is there a way to use this in VS2010?

Yes you can get this exact experience in VS2010. Just install the Azure SDK for VS2010 from http://www.windowsazure.com/en-us/develop/net/#.



Thursday, January 17, 2013 8:36:21 AM UTC
@Mohammad Sadegh Shad sorry to hear you are having issues with FTP. This is something which we are looking at.
Thursday, January 17, 2013 8:39:15 AM UTC
@Kostas web.config transforms should be invoked when publishing to Azure (both Web Role as well as Azure Web Sites).

If you are not seeing this please email me at sayedha[at]{microsoftDOTcom}.
Friday, February 22, 2013 4:21:39 PM UTC
The new features are great and so far it works like a charm within VS2012. The publishing profiles as well as the config transforms behave as expected - only our msbuild tasks seem to have a problem with it.

I get following error message from msbuild:

C:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Common.targets(609,5):error : The OutputPath property is not set for project 'MyProject.csproj'. Please check to make sure that you have specified a valid combination of Configuration and Platform for this project. Configuration='Debug' Platform='BWS'. You may be seeing this message because you are trying to build a project without a solution file, and have specified a non-default Configuration or Platform that doesn't exist for this project.

Tried adding the .pubxml - suffix and installing (& importing) MSBuild.Community.Tasks as suggested in this post without success.

Another attempt was to add /p:VisualStudioVersion=11.0 as mentioned by Sayed Ibrahim Hashimi in that post.

To be sure to be up to date I also updated to ASP.NET and Web Tools 2012.2 as published here.

My recent command looks like this:

msbuild absolute\path\to\MyProject.csproj /p:VisualStudioVersion=11.0;DeployOnBuild=true;PublishProfile=profile

Can anyone point out if I'm doing something obviously wrong?

Govinda
Govinda
Friday, February 22, 2013 4:29:43 PM UTC
BTW: The profile contains all the important properties I know of:
<PropertyGroup>
<WebPublishMethod>FileSystem</WebPublishMethod>
<LastUsedBuildConfiguration>Release</LastUsedBuildConfiguration>
<LastUsedPlatform>Any CPU</LastUsedPlatform>
<SiteUrlToLaunchAfterPublish />
<ExcludeApp_Data>False</ExcludeApp_Data>
<publishUrl>\\myServer\c$\inetpub\wwwroot\myWebsite</publishUrl>
<DeleteExistingFiles>True</DeleteExistingFiles>
</PropertyGroup>

So it really seems like msbuild is just not recognising/using the profile.
Govinda
Friday, May 03, 2013 1:15:19 PM UTC
I am looking at a way to generate one deployment package, with environment specific SetParameters.xml files based on the Environments configured in VS2012 or in the Publish profiles.

Ultimately I want the following
Package.zip
Dev.SetParameters.xml
Dev.deploy.cmd
UAT.SetParameters.xml
UAT.deploy.cmd
PROD.SetParameters.xml
PROD.deploy.cmd

So that for each check in we get one package, that can be moved from environment to environment by Developer, Tester or Support Staff dependant on an approval process.

I am interested in the PackageWeb scenario, but do not like the manual PowerShell step, we are looking for a One Click solution that can be implemented across environments.
William Bowsher
Sunday, May 19, 2013 3:33:29 PM UTC
Did anyone succeed in making VS 2012.2 perform web.config publishing profile transform from a cloud service project?
Wednesday, July 17, 2013 8:51:13 AM UTC
Hi Scott

Unfortunatelly your one line msbuild command:

msbuild MySolution.sln /p:DeployOnBuild=true;PublishProfile=Production;Password=poo

... does not pick up the Build configuration assigned to the Publish Profile. The proper way to go would be:

msbuild MySolution.sln /p:DeployOnBuild=true;PublishProfile=Production;Password=poo;Configuration=Production

Thats actually quite frustrating and should be fixed.

Regards
Mike
Mike
Monday, July 22, 2013 3:50:44 AM UTC
Mike - Does this help? http://sedodream.com/2012/10/27/MSBuildHowToSetTheConfigurationProperty.aspx This is outside of what we can change during a build.
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.