Scott Hanselman

What's the upper limit on the number of projects within VS2005?

September 26, '06 Comments [13] Posted in ASP.NET
Sponsored By

I had a friend whose team said:

"...most of them have 1GB of RAM and Pentium 4 3.0Ghz HT CPU’s. hey are all complaining about performance, especially with build times. Visual Studio is eating up most of the memory on their systems. Right now those with 1GB of RAM are generally running around 1.2GB of memory usage, so they are swapping like crazy and thus adding even more overhead. There are a couple systems with 2GB of RAM with Core Duo CPU’s. These systems are having the same issues where they are using 2.2GB of RAM, most of which is meaning eaten up by Visual Studio."

And I asked...

"Do they have too many projects in their solution?"

and he said...

"They said 50 projects and growing."

...and Scott Guthrie posted last week on optimizing build times in VS 2005.

"1.2GB RAM usage is pretty out there...50 large projects will strain a system depending on what they are doing. They should look hard at whether they really need all 50 in one solution.  Multiple solutions (even with multiple instances of VS running) work great too."

Clarifying Update from ScottGu:

You should be able to load 50 projects in a single solution - my comment in the origional mail wasn't to imply that you can't.  It was more than 1.2GB of RAM to-do so is larger than I would expect (unless each project is large and there are lots of cross project references).

Editoral: That makes me suspect Resharper as the issue in this case..

As an aside, they were also running Resharper at the same time, and it's currently unclear how much memory was being used by Resharper's on-the-fly analysis. I'll post if I learn more.

Nutshell: Try to break your projects up into reasonably sized subsystems that can build separately. Our SDK has about six solutions, each with about 10 projects.

If you're running VB and you're hitting the VB-specific problem with large projects, you can get a HotFix here.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Tuesday, September 26, 2006 11:59:15 PM UTC
I'm currently sick with the flu, so I'm a little more critical than I should be. But there's simply no reason any developer needs to be similtaneously working on 50 projects.

Seems like that's a complete violation of encapsulation. Please tell me we're not letting devs make ad-hoc changes in any of those 50 projects?

I'd say performance at this point is a BLESSING (and not at all blamable on VS 2005) as it might encourage some rethinking of overall process.
Wednesday, September 27, 2006 1:16:59 AM UTC
I just left a project in which the primary solution contained over 90 (yes 90) projects. :-\ Of course, as I understand it, (since I was a contractor working remotely) all the onsite devs had pretty underpowered machines so they had pretty major issues. Even on my dual core system with 2GB of ram, it still sucked, but I could still build the solution in under 5 minutes. :-D
Wednesday, September 27, 2006 2:38:14 AM UTC
The SLN I currently work with every day has over 50 projects in it; thankfully none of them are that large--most have around a few dozen .cs files (and some are much smaller such as a single assembly for each Web Reference). This SLN contains three separate EXE projects which have many projects in common. These EXEs used to each be in their own SLN, but that caused the build to break too often as nobody loaded the other two SLNs for every change; ever since combining them into a single SLN this hasn’t been a problem.

Putting these three EXEs into a single SLN also makes it easier as new projects are created: only one SLN needs to be modified rather than three.

What we really want is something along the lines of “nested SLNs”, SLNs could then be organized in logical groups (say all of the Web Reference assemblies) of about a dozen projects and those SLNs could be added to other SLNs. I’m sure there’s ways to make this work using MSBUILD, but we like using the Visual Studio IDE.

Wednesday, September 27, 2006 3:51:14 AM UTC
Resharper does have memory problems unfortunately.

Don't get me wrong - I love it bits and wouldn't be without it but it is memory hungry and can cause memory exceptions (although this rarely crashes VS - just pops up a dialog)

Cheers
Simon
Simon Hewitt
Wednesday, September 27, 2006 3:54:42 AM UTC
You should be able to load 50 projects in a single solution - my comment in the origional mail wasn't to imply that you can't. It was more than 1.2GB of RAM to-do so is larger than I would expect (unless each project is large and there are lots of cross project references).

One thing to check is the difference with and without Resharper installed (or any other additional package). It is possible that this is contributing to the slowdown.

Hope this helps,

Scott
Wednesday, September 27, 2006 8:43:07 AM UTC
I bet they have either dual core or hyper threading? Then the project system is clever enough to kick off two compiles at the one time, the downside is this takes twice the memory, simply turn off this option (or limit it to one compile - see tools options) and the problem will be solved!

Gordon
Gordon Smith
Wednesday, September 27, 2006 1:24:09 PM UTC
The solution that I am working on at work is approachin 80 projects. And yes, it is very much a productivity killer.
Wednesday, September 27, 2006 2:08:11 PM UTC
If we do not have all projects in a single solution, then we get problems with refractoring not working, as it can not find all the code that needs changing. It is also make it harder to manage the builds.

When I am working on part of the solution, .e.g one of the many applications it contains, I can unload the projects I know I don’t need.

It’s a pity that I can not have a named group of projects that I can load in a single step, or automatically unload all projects that are not needed by a given application. But still be able to build the COMPLETE solution when I wish to.
Wednesday, September 27, 2006 4:37:46 PM UTC
The memory consumption issue is DEFINITELY attributable to R#R (resharper); I've been using R#R in 2005 since the earliest EAPs for 2.0... memory consumption has gotten MUCH higher, but its capability for near-realtime refactoring is a strong mitigating force.

It'd be great if an MS dev got bit by the R#R bug enough to help them with memory aspects of perf, because the product is a real high value add on to Studio. (makes me glad I bought it back on version 1...)
Jonathan Choy
Thursday, September 28, 2006 6:15:34 AM UTC
When we have more than 20 projects in our SLN in more than 1 exe, we create separate SLN's. When we have 2 exe we create 3 SLN's because normally only the GUI would be different. The remark of J. Daniel Smith that the build breaks too often when using separate SLN’s because the developers don't open the other SLN's. That’s a problem we don’t have too much because we are using a continuous integrated buildserver, which will indicate you when your build is breaking for the other projects, so the developer can update their code. One remark our nightly build must remain unbroken. If this is broken, the guy who’s responsible for breaking the build must buy the rest of the team a beer ;-). But it's interesting to see what other people do and what problems they may encounter.
Friday, September 29, 2006 10:59:05 PM UTC
Scott,

I work on the Visual Studio Tools Platform team... We're very much interested in finding out whether the memory usage is due to Resharper or to something else -- I haven't seen too many instances where opening a 50 project solution should consume as much memory as you indicate.

Can you email me back at izzyg@microsoft.com -- I can forward to you a mini-tool that we've written to collect information on memory allocations. Your friend can then easily run this tool, and if they forward us the log, we can find out what is allocating so much memory and hopefully suggest some solutions.

Thanks,
-- Izzy

Izzy Gryko
Lead Software Engineer, Visual Studio Tools Platform
izzyg@microsoft.com
Izzy Gryko
Thursday, October 05, 2006 1:34:37 PM UTC
The general rule of thumb we use is a solution is a deployable entity that is made up of projects. In other words, if the deliverable doesn't rely on a project then it isn't in the solution. Even things like Common libraries are seperate solutions that are merely referrenced by the DLL in other solutions. We found doing it this way there was less breakage. Bottom line is a solution only has the projects that it needs.

I've seen solutions that were smart client solutions where web services, common libraries and everything else needed were inside of the main smart client solution along with everything else. A web service the smart client relies on is going to be deployed seperate than the smart client. So as a rule, we would treat the web service as its own solution. It helps to keep things seperate and allows different members of the team to work on different layers of the overall project (web services, asp.net documentation, front-end GUI, common libraries) at different times.
Thursday, October 12, 2006 3:49:43 PM UTC
Well, we have a significant amount of projects (around 80) in our so called "official" solution. The is the solution we use on our Team Foundation Server for the build process and under some circumstances on our local developer boxes.
As this is mostly unbearable for normal development, I have developed the following trick - called "stripped down solution":

1. Get the whole source code for the solution.
2. Build the solution. I use "msbuild official.sln" from the command line.
3. Create a new solution. I have an item in my Explorer/New menu for that.
4. Open the new solution and add just those projects you plan to develop in. Usually this is the production project and the test project.

Why does this work?
When the solution has been built, all the references between projects in the solution are pointing to the assemblies in the build output of the referenced project. This is accomplished by step 2.
When you remove a project from a solution all the references to that project are removed from the referencing projects. So we do it the other way around: We start with an empty solution and just add the things we want to change.
You might get worried when you look at the list of references in the Solution Explorer: some of the references show a warning sign. You can ignore that almost all of the time. Your solution will work as expected.

Some additional remarks.

Why the mega-solution?
-> It binds together all the visual studio projects we want to build at once.

Are there situations where the stripped down solution does not work?
-> Yes! When you rename an identifier, automatic renaming applies only to the projects within the current solution. So for that you will need the mega-solution again.
-> Adding project references. You obviously will need the mega solution to do that.

Do you check in the stripped down solutions?
-> No! The stripped down solutions are tailored to the developer and the task at hand. The stripped down solution I am using today will have little relevance for some other developer next week.


Maybe this will work with unloading the projects as well, but I have not tried this yet.
Ralf Kretzschmar
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.