Scott Hanselman

Dark Matter Developers: The Unseen 99%

March 7, '12 Comments [101] Posted in Musings
Sponsored By

In astronomy and cosmology, dark matter is a currently-undetermined type of matter hypothesized to account for a large part of the mass of the universe, but which neither emits nor scatters light or other electromagnetic radiation, and so cannot be directly seen with telescopes. - Wikipedia on Dark Matter

You can't see dark matter, but we're pretty sure it's there. Not only is it there, but it's MOST of what's there. We know it and we can't see it. It never shows up.

My coworker Damian Edwards and I hypothesize that there is another kind of developer than the ones we meet all the time. We call them Dark Matter Developers. They don't read a lot of blogs, they never write blogs, they don't go to user groups, they don't tweet or facebook, and you don't often see them at large conferences. Where are these dark matter developers online?

Part of this is the web's fault. The web insists on moving things forward at an rate that makes people feel unable to keep up. I mean, Google Chrome has upped two version numbers just in the last 3 paragraphs of this blog post. Microsoft probably created a new API and deprecated an old one just while I was typing this sentence.

Lots of technologies don't iterate at this speed, nor should they. Embedded developers are still doing their thing in C and C++. Both are deeply mature and well understood languages that don't require a lot of churn or panic on the social networks.

Where are the dark matter developers? Probably getting work done. Maybe using ASP.NET 1.1 at a local municipality or small office. Maybe working at a bottling plant in Mexico in VB6. Perhaps they are writing PHP calendar applications at a large chip manufacturer.*

Personally, as one of the loud-online-pushing-things-forward 1%, I might think I need to find these Dark Matter Developers and explain to them how they need to get online! Join the community! Get a blog, start changing stuff, mix it up! But, as my friend Brad Wilson points out, those dark matter 99% have a lot to teach us about GETTING STUFF DONE.

They use mature products that are well-known, well-tested and well-understood. They aren't chasing the latest beta or pushing any limits, they are just producing. (Or they are just totally chilling and punching out at 5:01pm, but I like to think they are producing.) Point is, we need to find a balance between those of us online yelling and tweeting and pushing towards the Next Big Thing and those that are unseen and patient and focused on the business problem at hand.

I like working on new stuff and trying to new ways to solve old (and new) problems but one of the reasons I do like working on the web is that it's coming to a place of maturity, believe it or not. I feel like I can count on angle brackets and curly braces. I can count on IL and bytecode. These are the reliable and open building blocks that we will use to build on the web for the next decade or three.

While some days I create new things with cutting edge technology and revel in the latest Beta or Daily Build and push the limits with an untested specification, other days I take to remember the Dark Matter Developers. I remind my team of them. They are out there, they are quiet, but they are using our stuff to get work done. No amount of Twitter Polls or Facebook Likes or even Page Views will adequately speak for them.

The Dark Matter Developer will never read this blog post because they are getting work done using tech from ten years ago and that's totally OK. I know they are there and I will to support them in their work.

* These people and companies all exist, I've met them and spoken to them at length.

UPDATE: I found a blog post from John Cook that mentioned Kate Gregory who also used this term "Dark Matter" to refer to programmers. It seems it's an apt analogy!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Wednesday, March 07, 2012 5:45:05 AM UTC
Important to remember that for the vast majority of developers it is just a job, nothing more. They go home and watch TV not read geek blogs or Twitter. Their not that worried about upgrading to .NET 4 because they didn't even know it was released yet. Just like 1% of bricklayers go home and read magazines on building better houses, most don't.
Craig
Wednesday, March 07, 2012 5:46:48 AM UTC
This is something I always remind myself when I'm flush from a recent community event like a code camp or DeveloperDeveloperDeveloper weekend. We'll be arguing over exactly how many DVCS repositories and what push/pull patterns should be used for what type of application or how to refine DDD with a mocking library. It's easy to sit there and feel smug about everyone "stuck" using (name your favorite old fogey tool - I think TFS is the traditional target). But, we're the 1%, not the 99%.

Most developers deliver real business value, day in and out, using chaotic or highly traditional (read: waterfall) processes where their success isn't measured by how cutely they can integrate the latest IoC Container but rather "is that report handling the latest acquisition correctly?".

Some times, it's really worth pushing your company to try out the latest agile thinking or web API. But it's also realy easy to underestimate how you could kill their productivity because they're not in a position to take advantage of the advantages and you just moved their cheese in a big way.

It's a bit like taking someone's minivan away and handing them a Lotus Elise. Yep, it's an amazingly fast car. But on a commute to and from work each day very few people are in a position to take advantage of it. On the other hand, they'll definitely notice the spartan interior, simple HVAC, and inability to see around the other minivans filling up the road.
Wednesday, March 07, 2012 5:51:48 AM UTC
MS Access is still the best database in our environment. NO WEB environment...
Wednesday, March 07, 2012 6:09:52 AM UTC
Nice article. I enjoyed reading it as I can relate to being a bit of both. I'm currently working in MVC3, Razor, jQuery and working with CSS Flexbox that only works in Chrome Canary and our in-house custom CEF browser implementation.

But...

I feel I was still more productive when it came to business functionality delivered when working in the AS/400 environment as a dark matter developer. Using Synon/2E and RPG. Stable, robust, proven, ancient.
Wednesday, March 07, 2012 6:23:58 AM UTC
I almost feel this post needs to be read with the x-files tune playing in the background ;)
Wednesday, March 07, 2012 6:28:30 AM UTC
Your point is acknowledged, and it's all well and good until they're promoted to enterprise architect.
Peter
Wednesday, March 07, 2012 6:36:55 AM UTC
Just a small side note: "You can't see dark matter, but we're pretty sure it's there"
Actually nobody's sure that there is dark matter at all. Not even those vocal "scientists". Dark matter (and some other exotic entities) are just mathematical constructs invented so that the Big Bang theory stands.

But I agree and like the article about unseen developers that get things done.
Daniel
Wednesday, March 07, 2012 6:39:20 AM UTC
A second note, after re-reading: You mention that you try to remember the dark matter devs when working at Microsoft. That's wonderful.

Similarly, I've been fascinated by Ayende's posts on architecture where he eschews repositories, "traditional" dependency injection, and everything else we've been told is absolutely necessary in favor of an "enterprise architecture" featuring a grand total of 8 things. It's refreshing to see someone as prominent as ayende promoting a style that is simple because he has eliminated as much overhead as possible. It's wonderful to be able to read and understand his code, which I can pick up even just reading it inline in his blog posts.

Anyway, I like the kind of simplicity that happens when your framework/architecture is small enough to be understood.

It's late.
Peter
Wednesday, March 07, 2012 8:33:44 AM UTC
Saw this TED video last night and it seems to correlate:
http://www.ted.com/talks/susan_cain_the_power_of_introverts.html

There is a vast range of developers (as in how outspoken they are online). I wonder if there is a relation between being extrovert/introvert and the online-presence that one creates. I would assume there is a strong correlation. Extreme extrovert people would tend to be on twitter, facebook, linkedin even google+, their blog, github, ... and be present with a lot of 'noise'. On the other end of the spectrum I suspect extreme introvert people to bare register online. I have met some people like that and they do not even know there is a site like stackoverflow for example.

As in the TED video, the best is to acknowledge ones own position (extrovert vs introvert) and acknowledge that either 'side' has their benefits. Together we are stronger.
Wednesday, March 07, 2012 8:36:10 AM UTC
It does make me a bit sad though, that this would need reminding, speaks for the self-absorption of online communities.

Daniel, i think dark matter hasn't got much todo with the big bang theory, it is a construct to explain the discrepancy in the gravitational structure between eg a galaxy with only its luminous mass and what is actually observed. Short of changing the theory of gravitation, dark matter seems a not so bad hypothesis.
Wednesday, March 07, 2012 8:52:29 AM UTC
I've met quite a few "dark matter" developers, throughout my days as a dev, and from what I have gathered there are mostly two types.

First type, developers that find a (good) job that they work long hours, and produce software for a specific app/suite. They are too busy working on it while being rewarded for it, so they are too tired to "look around" and don't really care. They like their job, but do not have an "actual need" to learn about new stuff.

Second type, developers, that have become devs, not because they actually wanted to, but rather because they saw it as one more career option, in the same way that they would become finance analysts, or marketing execs. Once they get a job that pays the bills, they are content with it and do not care what goes on in the software world. As Craig mentioned, they go home and watch tv...
Wednesday, March 07, 2012 8:58:59 AM UTC
You are perfectly true - only in our city there is an estimate of over 500 professional developers using .NET in their daily jobs.
Despite this, to any conference and presentation on Microsoft and .NET technologies, you won't see more than 60 people (among which probably 40% are students), no matter if the event is organized by the local .NET groups or Microsoft itself. Even among those who come to a presentation, many will come only once out of curiosity, then never again.

In the online forums it's the same - no more than (the same) 10 - 20 people are active.
As far as I can tell, countless developers seldom read a blog, never user twitter, and very, very few have an active blog of their own.

For many, the latest and greatest technologies are great, but they don't matter as long as they have to develop daily, 8 hours/day, a huge WinForms application started only 3 years ago.
Wednesday, March 07, 2012 9:27:22 AM UTC
Having read your excellent blog for many years, this will be the first time that I've commented (something must have struck a nerve!).

I agree that there are many dark matter developers out there, but I just wanted to give, perhaps, a different take on some of them. I consider myself a dark matter developer, but not because I don't read blogs or attend (many) conferences.

I think of myself as an above average developer (purely based on the number of tests I've taken for interviews. Can't say I'm a super developer based on the test/criteria you put in one of your blogs), who has a voracious appetite to learn everything new. To consolidate what I've learnt, I need to practice, and naturally, work time is not conducive to this, as I'm there to work, not play (unless you're either very fortunate that you have little work to do, or you have an accommodating boss who gives you R&D time), but, because like a lot of people, I have a 3 hour round trip commute every day, which leaves you little time to practice. To be honest, although I've very grateful for your fascinating tidbits, I don't really understand how have the time to play & blog, when you appear to run a very busy department. Hat tip to you and everyone else who can manage it. In the end, you have to realise your limitations and just do your best.

As for blogging, why would I, just to re-gurgitate stuff which has been said a million times by people far better placed than I. And not every one is a natural writer, you should be thankful for the gift you have. As for Twitter, for me that's just noise, and there's too much sifting the wheat from the chaff; can't be bothered. Anyway, RSS is excellent. Facebook? Don't get me started; assuming there's 800 million FB users, I'm more than happy being one of the ~5.2 billion not on FB, having my privacy eroded.

There was a time when I tried to keep up with a lot of what people were saying, but feeling constantly frustrated that there wasn't enough time in the day to read everything. In the end, I just had to choose to follow people who chime best with me and except that you're going to miss a good deal of information.

There, that's off my chest. But do keep up with your excellent R&D and blogging.
Robert S
Wednesday, March 07, 2012 9:53:22 AM UTC
Read DailyWTF to get a glimpse of this dark matter.
In Russia, there are even more "dark matter" coders, because IT blogosphere is very elitist and topics discussed there are mostly "rocket science" like Haskell (or something more complex like Agda or Coq).
Wednesday, March 07, 2012 10:32:54 AM UTC
Two thoughts on this.

First, I would like to be more like them. Every time I sit down to work on a side project at home I find myself updating all my libraries to the latest and greatest and doing very little actual productive work.

Secondly though, some of these dark matter devs are writing code that is downright horrid, and I've had to debug and fix some of it. For some reason there is a strong correlation with VBA, though it can be written well. There is a reason for new fangled patterns and technologies, and usually they help to make maintainability and readability better.

(I have a sneaky suspicion that web things will stabilise soon on one-page sites using web api but with simple fallback to multipage, a bit like project silk. Looking forward to seeing what you do with the single-page stuff. EF Code first or the fluent NHibernate equivalent seem pretty stable already.) That wasn't a third point at all.
Wednesday, March 07, 2012 10:54:31 AM UTC
So true.
I can't compete with an AS400 RPG programmer in productivity.
And the users are more productive in a greenscreen app with function keys than a fancy WPF app.

But what I "hate" now is those programmers doing OVER designing. X patterns in a stupid CRUD application to do 2 steps. A composite pattern, with state and with a decorator and ...
Just keep it KISS and clean.
Wesley Walraeve
Wednesday, March 07, 2012 11:01:21 AM UTC
Our customers don't care that their systems aren't written using the latest technologies, they just want them to keep working like they have worked for the last 7 years. They don't care about EF, they don't care about Rx and they aren't going to pay for us to take the time to upgrade our toolset unnecessarily.

As I sit here writing code in VS2005 and SQLServer 2000, it's nice to know that I'm not forgotten.
Wednesday, March 07, 2012 11:20:22 AM UTC
@Dr Herbie

But maybe they would care if they understood that it could help them have new features in half the time/cost (after the initial pain of upgrading). Though again the risk is that you endlessly spend money updating and never actually give them any features.
Wednesday, March 07, 2012 12:30:52 PM UTC
I am really afraid about people that just are getting stuff done.
I think that this same people write code that works, but dont write code that continue to work.
Write something that works now is very easy, but something that is maintainable is when we see the difference between good and bad programmers
Wednesday, March 07, 2012 12:40:25 PM UTC
@Mark Adamson

We tried that and unfortunately a cost to this quarter's budget to save on the next quarter's budget is a concept that a lot of our customers have a problem with.
Our technical debt is through the roof, but our customer's aren't prepared to pay for us to do the housekeeping (as they pay us by the hour they keep tight control of how we spend our time).
Wednesday, March 07, 2012 1:00:58 PM UTC
So what category do developers fall into if they work at a forward thinking dynamic company but they choose not to join the community?

Belbin says you need various types of developer to make a high performing team. But stagnating developers could add risk to a team/project. They require more training, are less likely to get things right and often require carrying forward.

Wednesday, March 07, 2012 1:31:46 PM UTC
I'm probably one of the dark matter guys, but I'm not stagnant. I actually read and research topics and read many of these blogs I just don't have time to participate (or maybe don't think I have any thing to contribute). After working 50 hours a week and having a wife, kid, honey-do list, and just enough at the gym to not piss my doctor off, there's not much time for other stuff. The guys I work with do research and development too we just don't do it on developer tools or general CS technology that can be discussed at length on the web, but in industry specific domains where either it's too specific to our industry and/or IP constraints.
Wednesday, March 07, 2012 1:45:31 PM UTC
Where I work there are many talented developers, way more than myself; none of them are online. What I try to do as a blogging citizen and general member of the community is to bring in information I learn from them. Even the ease of Stack Overflow isn't enough to bring them in, despite the fact that there's a 50% chance while you're walking around our office SO will be on someone's screen. They're like the equivalent of forum lurkers. It doesn't mean they're less passionate about .NET, it is what it is. They do contribute to an internal wiki; much of it is documentation for internal components but some have wrote architecture guidance and things like that.

The people I work with have done really cool things that probably no one knows about:

* Built our own app store for Android/iOS for employees

* Built single-sign-on for Android/iOS using our AD infrastructure, both native and hybrid apps

* Lots of SAP to .NET magic for front-ends

* Custom building of iOS/Android applications using SSH/web services from our build tool (Windows) and Mac Mini (I did a lot of the SSH stuff)

There's just a lot of stuff that these guys & gals do that stays inside and no one knows about.
Wednesday, March 07, 2012 1:58:02 PM UTC
I think it's odd (and a little condescending) that you assume people who don't write blogs and forum posts also don't read them. At the moment I read a great deal, but don't write much, because I'm in a learning phase. I won't be writing much until I've got something to contribute. (I realize that many prolific posters aren't similarly constrained.)
Thing is, as a producing programmer I've been in a learning phase for roughly 80% of my career, which started with a beta version of Windows 1.0. And as a producing programmer, I need to learn a lot of non-programming stuff at the same time. NerdDinner, AdventureWorks and the like are good teaching tools, but they don't begin to capture the morass of competing requirements and demands that real-world developers struggle with. Finally if I'm successful on a given project, I usually have to stick with it for a few years before I can move on to the latest and greatest.
So I'm perpetually behind the curve. I once heard a measure of mastery that went like this: if you write down everything you know on a subject, how big would the book be? At one point, I could have written a pretty good book on C++, but by the time I reached that point, there was no need for another book on C++. Currently, I'm neck-deep in technologies that I couldn't produce a decent pamphlet on, as has usually been the case.
Which is why I keep reading. By the way, Scott, your stuff is usually pretty decent. Thanks, and keep it up.
Andy Lipscomb
Wednesday, March 07, 2012 2:04:04 PM UTC
Intriguing thing to remember. In my company, the vast majority of our developers are "Dark Matter" devs. They get their assignments, go to their cubicles, and they're unheard from unless you meet them in the bathroom or on lunch. It's also interesting from my perspective how different departments can be within a single company. All of the devs in my area try to stay up to date either at work or by doing projects and blogs at home, etc. There are other departments where every single developer is a punch card employee working with archaic languages. They DO get stuff done though.
Wednesday, March 07, 2012 2:22:00 PM UTC
Scott,

Just to add to what Robert S. said -- I am also an active .NET developer who seeks out the latest libraries, patterns, and techniques. I try to keep on top of what Microsoft and others throw out each year. I try to listen to .NET Rocks, Hanselminutes, and anything else I can to make sure I'm aware of what is out there to help me with the applications I build.

But I don't have a Facebook or Twitter account. I don't have a blog and I have zero online presence. And that is by choice. I think there are a lot more of us than you think. For me, personally, I do not have time to engage with social networks and I have even less desire to do so than I have time. I'm not an introvert by any means -- I just have no desire to become "Internet famous" or be able to Google myself and see my footprints everywhere.

I hope that doesn't sound like a slam on you. I wasn't being specific. You do what you do because you're great at it and it is something you feel compelled to do. I would rather spend my free time with my family, playing the guitar or writing applications for side businesses that I have. So maybe we need a category for people like me. We're like pulsars I suppose. So now it's time for my light to go off again...
Wednesday, March 07, 2012 2:26:22 PM UTC
I think I lean towards dark matter, but I have a torch that I turn on occasionally.

I get a little annoyed by devs who are always shining the damn thing all over the place, they remind me of crows who are after shiny baubles all the time.

Just get stuff done. Play at home, and then when you've discovered real value, bring it. Until then, please refrain from littering those damn baubles all over the place. I hate seeing good opportunities turn into someone's science project. It's actually a little irresponsible at times.
Wednesday, March 07, 2012 2:29:04 PM UTC
"The Dark Matter Developer will never read this blog post"
Maybe many of them will. Just because they don't comment doesn't mean they don't read either. I know many of my colleagues follow blogs, but they don't comment them. I myself often read your blog (and a few others) but never (or extremely rarely) write any comments.
Must be my personality :)
Wednesday, March 07, 2012 3:41:50 PM UTC
I fit the profile of an introverted developer who just gets things done, however some of the other comments paint a pretty negative picture with which I disagree.

- Experts in technology are often introverted, and will therefore usually not participate in the usual social mechanisms. (e.g. I can't even relate to or understand why anyone would want an iPhone, Android, or WinPhone.)
- They are often deeply knowledgeable about a range of tech and non-tech topics, but it takes years to gain this knowledge. Unless you've spent 10+ hours a day for 5+ years working exclusively with some tech, then you are NOT an expert!
- Experts don't write books. They spend 5 years designing and implementing a product, then come up for air, and find that those in power have decided to completely change everything without even consulting them about what was right/wrong with the previous technology.
- Being an expert is not a good feeling. You know you're an expert when your are stuck trying to figure out some horrendously difficult problem, and you realize that there is no one who can help, because you know a particular area of technology as well as anyone. If you can't figure it out, then there is no wise guru or mentor to come to the rescue. There are no smug contented experts.
- Experts strive for simplicity, but a problem with experts is that the smarter they are the more inscrutable their thinking is to lesser mortals. Microsoft probably has many experts, but I suspect they're smarter than me, because I don't find many of their designs simple enough.
- I prefer humble experts. The proper attitude is "Oh no, I'm smarter than everyone else. I hope nobody notices.", and "Although I can tell, unfortunately, that I seem to understand things more deeply than anyone else, I also notice that I'm still an idiot who can't understand anything as deeply as I should.".

Disclaimer -- I'm not actually an expert in most things. Most of my career has been designing and implementing software that takes 6 months to 2 years from start to finish. This is not enough time to reach truly deep expertise. But I *have* spent the last 5 years working on a technology that I hate (Java/Swing), and I'm almost starting to feel like an expert with that. I've been starting to suspect that the grass is greener on the VB/WPF side of the fence, but MS will probably kill off that technology before I have time to find out.

Microsoft (and others) need to do a better job of coaxing information out of the actual experts in their technology, because they consistently make avoidable mistakes. I've been playing with VS11 beta lately and noticed a number of them.

For example, removing the macro feature is a classic case of misunderstanding the data. Although few use the current feature, this doesn't mean that a macro feature is unnecessary or unwanted.
Here's the real history...
Visual Studio had a decent macro facility that could be used to solve real problems. The key feature was that it was easy to use, and didn't require loads of time to understand a complicated API. This macro facility was discarded in VS7 in favor of a "more powerful" one that was orders of magnitude harder to use. Now in 2012 that decision has finally caused the useful feature to be killed off completely.
Wednesday, March 07, 2012 3:44:24 PM UTC
Lurkers of the world, UNITE!
Wednesday, March 07, 2012 4:19:44 PM UTC
There will always be people who are satisfied with the technologies they are using daily. If you are one of those people, then that is great. However, we are all in a technology field. Technology matures and is obsoleted. If you are a 'dark matter developer' you still owe it to your self (and your family) to understand the market conditions. If you are a technology 'ABC' expert and making a living then hats off to you. Being an expert of technology 'ABC' also means you should be able to predict when 'ABC' will no longer be a viable solution to your employer (and no longer pay the bills). The knowledge that 'ABC' will be obsolete in 6 months, usually means that you must keep at least a high-level view of the related technologies. While blogging, participating in social circles and such is not necessary we should be encouraged to stay informed.
Wednesday, March 07, 2012 4:21:13 PM UTC
It's too bad more people in the industry don't recognize this. Job postings are always an alphabet soup of the latest acronyms and version numbers, then you get in and find out they've still got a ton of Classic ASP that they're maintaining. It's easy to feel like you're getting left behind when every day there are posts about jQuery, Backbone, MVC, Nancy, Micro-ORMs, etc, etc, etc. Maybe the 1% needs to just chill once in a while and give the rest of us a chance to catch our breath.
Wednesday, March 07, 2012 5:24:43 PM UTC
I am one of those "dark matter developers". Although I regularly read developer blogs and so keep up to date in that way, I don't get much opportunity to use this since our product is about 80% VB6 (the rest is .Net). It's a very large product and we're a small company so converting the existing code to .Net is really out of the question. Although I read as much as I can to keep abreast of new developments, I don't get much opportunity to practise, so I don't retain as much as I'd like.

The speed at which the technology changes is, I think, key to moving forwards. As .Net has evolved, so have the various methodologies and associated technologies, and I struggle to find recommendations and guidelines that I can trust, particularly as there always appears to be some infighting involved. So I've hit a kind of analysis paralysis where I don't dare make a move in case it's in the wrong direction. Taking the Entity Framework as an example: firstly there was a big buzz about ORMs, then the Entity Framework appeared and was rubbished in some quarters of the internet as not being a proper ORM, but that has now progressed through various versions so I find myself asking such questions as: Is it still rubbish? Was it ever rubbish and the people criticising it were wrong? Who can I trust? Another example: recently there's been a series of blog posts where Northwind Starter Kit from the "Microsoft .Net: Architecting Applications for the Enterprise" book has been heavily criticised. This is now quite an old book, so is it still worth reading, or is it out of date? Was it ever worth reading, given the criticism that the NSK is getting? As an outsider reading the arguments going on, I am unable to decide who's right. Are these pragmatic arguments, or are people just arguing for some academic reasons that have no bearing to the real world? It's a question I can't answer without being fully able to understand the problems that these solutions do or don't address.
Wednesday, March 07, 2012 5:25:54 PM UTC
Maybe I'm not so much a Dark Matter "Programmer" as a Dark Matter "Lurker". Scott, I love your posts and read them probably more than any others (really!). You certainly are a prolific online presence.

But respectfully, I don't agree with the dinosaur-like characterization of those (like me) who don't blog, tweet, facebook, or indulge in other online forms of digital flatulence. (True, it's not all like that, but the other side of this equation is that far less than 1% of online content of this nature is worth the bandwidth that each byte of it occupies. Indeed, we need better tools to sort the garbage from the gold, as it were.)

I don't believe that I fit the archaic mould you describe. I do web development work for large corporations (indeed, my work history shares an employer with yours!), often using bleeding-edge tools and techniques, but most often I need to balance the use of these tools with technical risk. .Net 4? Still a bit early for the majority of corporate environments, IMHO. I have no doubt that will change very soon, but there's simply no time for me to mess with that stuff when right now there are tangible things to deliver that need to instantly integrate with exiting platforms. Software vendors are so adept at making everyone feel that what they are currently developing is instantly obsolete. The process breeds an insidious wave of attention deficit disorder. We can barely begin adoption of a new technology without being bombarded by pundits who tell us that our plans are already out of step with the times.

And while I celebrate the ability of many of the current social media to bring people together, retain links with your deep, dark past, or whatever you wish, I fervently believe that there is a difference between cyber-reality and, well, reality. In other words - not to put too fine a point on it - some of us simply have a life. And for many, these social tools turn them into, well, social tools. (Sorry, a cheap joke but I couldn't resist.) How many times do you go out to a restaurant and see couples sitting together, tweeting and texting others on their mobile devices? They have no sense of the here and now.

IAC, thanks for a provocative post. Keep 'em coming.
Wednesday, March 07, 2012 5:33:45 PM UTC
Patrick - Great comment, thanks! I would disgree with a few things you are saying though. I don't think I characterized folks as "archaic," in fact, I said that you all were getting things done while we were moving bits around. Second, the idea that "some of us simply have a life" does imply that others (myself, etc) *don't* have a life. I've got a wife, two kids, a mother-in-law that lives with us, a full time job, and a blog and two podcasts on the side. But I take my kids to school every day, have sit down dinner with them every
and read books to them and tuck them in every night. There are non-social folks that would be hard-pressed to say the same thing, no?

There's no judgement express or implied in my post, just simply an acknowledgement of the existance of different kinds of folks. Your thoughts?
Wednesday, March 07, 2012 5:34:31 PM UTC
As some other commenters pointed out, unfortunately, the programming is not a passion for most people these days, and I see that as the main reason for what you mentioned which is absolutely right.

The problem is that computers started by a small group of passionate people who wanted to make a better world for others and they spent best years of their lives at a young age building software when there was no tooling and libraries to assist them. At that time, programming wasn't a prestigious and well-known job with a very high income even more than man other specialized jobs.

Thank to their efforts, we're living at a time when computers are very common, and all the businesses need computer-related tasks which require programmers to write code. On the other hand, many companies (especially Microsoft) have made the task of programming super-easy for many common cases. Therefore, a person without much of a background in principles can write many of the data-driven applications using these tools without caring much about the community and practices. Such a person is not passionate about what he's doing, so as long as he's *just done* with his responsibilities to get user's data and display them back to him/her, he's satisfied, and can go home have a life like others.

To be quite honest with you, this is a very frustrating point for me. Not only in the context of the programming jobs these people have flooded the community with a negative fingerprint making it harder for passionate people to breath, but also you see such motives among the youth for entering the computer-related majors at universities. Everyday, I witness many junior and senior college students who entered this field for money/career reasons without having any passion. Upon graduation, they're unable to write simple programs.

Just my two cents. Thanks for your post :-)
Wednesday, March 07, 2012 5:48:46 PM UTC
I would consititute a dark matter dev. Working on line-of-business applications means that speed/accuracy are way more important then cutting edge. I read to keep informed but find that new things are often dangerous - they are often fads that fade quickly and then don't provide documentation about the part you are working on.
There is a constant business demand for VBA and budget constraints (almost 0) mean that C# 4 and mysql fill in the remaining gaps.

My note of advice to Microsoft : big tech stacks are good for big projects and teams. A small stack is equally useful for developers churning out LOB apps that may only get used by 10 people - this is why winforms works so well still.
john m
Wednesday, March 07, 2012 5:53:38 PM UTC
I think if people are reading this blog, commenting on it, or even lurking then they are still participating. Maybe it's passive, but they're not in a locked box keeping to themselves.
Wednesday, March 07, 2012 6:15:10 PM UTC
The problem is when Dark matter developers get promoted via the Peter Principle. You now have people who are expected to make decisions - who use the same old tools at their disposal to make new decisions, overlooking and or overriding other, more knowledgeable people - but perhaps newer people in the process. Not to mention the mess of code they leave behind each time they move (positions or jobs).
Wednesday, March 07, 2012 6:40:51 PM UTC
The example you gave with programmers working on embedded devices absolutely makes sense.

Then there are some areas where you can't afford to be left behind. I think web development is one of them. And it's not just about using the latest and greatest server-side technologies. It's also about keeping up with browser trends, mobile trends, how your audience changes over time, etc, etc.

Something that has changed drastically over the past decade is web developers' understanding of web performance (at least for those of us paying attention). I believe you interviewed Steve Souders at one point. He's been instrumental in getting the word out. And his YSlow tool is really cool too. Any web developer who doesn't follow blogs or really try to keep up with things could easily miss out on little gems like this.
Wednesday, March 07, 2012 7:07:37 PM UTC
I guess I would consider myself in the "grey matter" category. A lot of us are there. I read blogs, install betas and love to try things out, but I don't incorporate all this stuff into my work right away, nor do I re-do my working code unless there it is totally necesary. I don't have time to write blogs, and live outside the us so I can't go to conferences often. But we love what you 1% do and we hope you continue to do it.
Manuel
Wednesday, March 07, 2012 7:10:18 PM UTC
I am a lot like Sean above. I am likely one of those dark matter developers. I spend a lot of time writing, researching, reading, but I don't say that much because I feel I'm always still learning and not in a position to say something which may have already been said. I spend some free time coding for fun, learning new ways to code. I am a father, a grandfather, have the honey-do list, am blessed with a programmer for a wife so I can cuss and discuss things with her, but we also are advocates for special education and work in the community in other ways. So I don't typically weigh in unless I feel I have something profound to say. I tend to remember Samuel Clemens' maxim; "It is better to remain silent and appear a fool, than to open one's mouth and remove all doubt!" I would rather read and learn from everyone and contribute when I feel I've learned enough to be more than dangerous.
Wednesday, March 07, 2012 7:24:34 PM UTC
It is true that, there are 99% dark matter developers, it is something like Yang - Ying thing, light depends on darkness and darkness depends on light. Imagine if there 100% dark developers then picture above would have been #000000 or imagine if all developers would have been like shinning stars then picture above would have been #FFFFFF, in both case's life would have certainly sucked.

I think it's natural, the way the dev universe is or should I say that's what makes it beautiful. Some's shine and some help shine (dark matter dev's).
Wednesday, March 07, 2012 7:42:07 PM UTC
In my experience, there are a lot of programmers (average, bad and great), may I say the vast majority, that just aren't as enthusiast. They know how to program, and they'll learn new stuff from time to time, but they aren't into blogs, and rss feeds, and following people in twitter etc.

They use the internet to find answers when they need and thats it. But they usually are just not that much online, not only do they not follow technical blogs, they don't follow blogs at all, or use twitter at all.
Wednesday, March 07, 2012 8:58:24 PM UTC
Like Patrick, I too felt a little of the "respect your dying elders" message in the OP, though it was not explicit and I'm certain was unintended. Truth is (for me at least), it stems from insecurity.

Not all dark matter employees are fulfilled by a paycheck and some "code that works"; many of us do have aspirations and progressive desires, and are embarrassed or dissatisfied about working in a shop spackled together by WinForms, SHTML includes, and PowerBuilder lines. They want to get out of their glacial-speed public service positions, but feel light years behind the nerds that tweet about their MVC4 Web API. These things contribute to the "<a href=http://www.hanselman.com/blog/ImAPhonyAreYou.aspx">phony vibe</a>" you brought to light so well back in August, making it hard for some to grow-a-pair and take risks.

Add to this lack of fulfillment/confidence the speed at which the IT world moves, and we begin to spend the majority of our career trying to stay on top of our careers. This scenario is arguably the worst of both worlds, as you're not the guru everyone wants at their startup, and you're no longer the trusted productivity guy because you've been putting the smack down on so many tutorials and code exercises.

We hear from our mentors and heroes online, "build some Stack Overflow rep", "get a blog rolling and show people what you got." Some lurkers give it a shot... then are quickly discouraged when their SO questions is down-voted for being too stupid. Not a very welcoming community.

Qerim said it well, the pundits of modern technology and programming theory (yourself excluded) often come off as "crows after shiny baubles"; or better yet, bluejays that just want people to know they are there and get some attention. Knowing well, not many of us are the Scott Guthries, Linus Torvalds, or John Resigs of the world - do we really need more chirpy birds?

Then there are those who say "the problem is when dark matter developers get promoted via the Peter Principle", or "they're fine until they get promoted to enterprise architect", or "they write code that works but doesn't continue to work".

I apologize for ranting, but I there is a point under it all and brings up questions for discussion:

1. How does a developer minimize and/or deal with being stretched over the log? (i.e. losing his/her productivity whilst investing time and money in training efforts)

2. How does a developer participate without being another chirpy bird?

3. How should a developer without much of an online presence go about networking with the "guys in the know"?


Thank you Scott, for another informative and provocative post.
Wednesday, March 07, 2012 9:04:40 PM UTC
@Tony Dunsworth - Well said. I wish I could write half as clear with your brevity. :)
Wednesday, March 07, 2012 9:08:44 PM UTC
Why is important to that developers to get online?
Wednesday, March 07, 2012 9:24:00 PM UTC
I think there are way too many assumptions being made here. The first is that a balls-out-uber-community-leader like Scott can't also have solid work-life balance. I flatly reject that. I think that's a choice. (I also think that Scott doesn't give himself enough credit in terms of family, but I only know him via FB and former loose reporting lines at work.)

I also observe that those who are "dark" and don't keep up are solid producers, but I also don't believe they push technology in a way that makes it more efficient. Work for any huge bank or insurance company and observe the mounds of COBOL programmers still necessary.

What I think I'm getting at here is that both ends of this spectrum are, in my opinion, not working toward the "right" goal. That goal is to leverage technology to do "stuff" in a faster, more efficient way. Less work, more result. That's the world I dream of. I love the tech, I blog (occasionally), I do the open source thing, etc., but when I'm done working, I want to hit the hot tub or plan my next trip to Hawaii, or just wrestle my kid... because I used tech to make that time.
Wednesday, March 07, 2012 9:40:38 PM UTC
"I mean, Google Chrome has upped two version numbers just in the last 3 paragraphs of this blog post."

Priceless. It's also more than a bit annoying. I have a computer at home that sits in my attic and does... stuff. I remote into it every couple weeks or so for some reason or other. It's always asking me to upgrade Firefox, install windows updates, update Flash, blah blah... I mean, can't it just be happy to hang out and get old?

But really, as someone that tries to keep up, I've realized how completely futile it is. You can't possibly be an expert on the cutting edge, even in a narrow area of focus, and get your work done. At least I haven't figured out how.

I accept that there will be some balance. I will not always be as cutting-edge as I would like. I will spend effort learning new, better things when I am dissatisfied with something old. But if I am using something that works and is not overly burdensome to use, I will try not to care if it's old, and I will promise that I won't replace something functional with something newer and slicker just because it's newer and slicker.

Because I don't have time.

Actually that's a bald faced lie, I waste tons of time learning about new things even when the old one is good enough. But I try not to do that when there's work to get done.
Thursday, March 08, 2012 12:18:01 AM UTC
"[Dark Matter Developers] don't read a lot of blogs, they never write blogs, they don't go to user groups, they don't tweet or facebook, and you don't often see them at large conferences."

I don't know where I belong there. I read blogs, I use FB socially when I have time. The rest, I don't have time for. In the last year, I have started new projects using Silverlight, Entity Framework, and MVC. But I still support client apps I wrote in ASP.NET 2.0 and, God help me, VB 6.0 (it is at least on the way out - replaced by Silverlight, another soon-to-be has-been). My business is about "get the job done" but also "support and enhance the job done". Most of the time, that doesn't include changing frameworks and retooling the app.

I don't complain about any of this. I do my best to stay fresh because each new technology makes my job easier and keeps my job fun. I love this field.

Thank you, Scott, for the nod in my direction. And thank you for helping to push the technology forward to the next new thing. I think the world needs the 1% and the 99%. :)

Ray Humphrey
Thursday, March 08, 2012 1:03:19 AM UTC
I think there is a middle group, let’s call it the “gray matter” group (not dark, just dull). Those, like me, who attend the user group meetings, use the tips from you, stack overflow, and other online resources. Always learning and looking for what technology will be in our future but trying to weed out the noise of ideas that never get traction. But, I have never contributed a single community solution nor do I have a blog post. You would not see me online, but I do lurk in the background. Part of my justification for not contributing is that I do not have that gene that makes a teacher: someone that can simplify a complex idea down to easily digestible small chunks of knowledge. So rather then add to the noise, I listen.

Look forward to meeting you at the Dallas Day of Dot Net
Thursday, March 08, 2012 2:21:45 AM UTC
Scott, thanks for your reply. Please forgive me...my words came on a bit too strong and I should have chosen them more carefully. I didn't mean to tacitly imply that you didn't have a life. Indeed you seem to be one of those energetic people who can manage it all, and more power to you! And it is in fact through blogs like yours that I do try to keep track of what is happening.

If anything, the essence of what I'm saying is that people like myself just can't keep pace with everything and have a life as well. I too have a wife and children, and my guiding principle is that "I work to live; I don't live to work." (Although sometimes this isn't much in evidence given the amount of time I spend at the keyboard.) IAC for me, as much as technology is what I work at, I really don't feel that it defines me per se.

And then, of course, we all need to (and should) make our own value judgments on the impact on our lives of technology...hence my "social tools" quip. But I wasn't targeting anyone here, just projecting how I would look upon myself hypothetically. That probably says more about me than anything.

Sorry for any inferences!
Thursday, March 08, 2012 2:34:05 AM UTC
@Patrick - My intention was much the same as yours, describing the feeling I and some of my peers share, not commenting on Scott or any others who are so badass that they can do it all without going bananas.

Some days I'm totally stoked and crank away learning, other days I feel like I'm on lap one as people are approaching the finish line.

I really liked what you had to say - especially the last thought about measuring the impact of technology on our lives and our responsibility to make our own judgements accordingly. A broad statement, yes, but I like your point.

Take care.
Thursday, March 08, 2012 5:05:17 AM UTC
Thanks Trey. I empathize with your sentiments as well. Folks like us have to stick together!
Thursday, March 08, 2012 6:01:40 AM UTC
mmmh; Embedded (Atmel, PIC etc) I will very likely write in plain old C (not C+ nor C# - not that I don't know enough about either to use one of these instead; Show me a real advantage...)

Website I will write in the server's local scripting language but I prefer PHP.

Games (for PC) I used to write in QB (tho I never published a QB one) but wrote my last one(s) in VB6.0PE

Maybe I am one of your "dark programmer/developers"


BUT: I read (your) blogs and have at least some inkling of the latest developments in (more than just) the programming world so...

Maybe I am not :p
Thursday, March 08, 2012 1:01:35 PM UTC
Just a small side note: "You can't see dark matter, but we're pretty sure it's there"
Actually nobody's sure that there is dark matter at all. Not even those vocal "scientists". Dark matter (and some other exotic entities) are just mathematical constructs invented so that the Big Bang theory stands.


That just isn't true. There are quite a few lines of evidence for the presence of dark matter. It is revealed by some gravitational lenses and the speed the sun orbits the milky way. Also .... most of the missing stuff is dark energy, not dark matter.

Just because people don't post blogs doesn't mean they are not enthusiastic -- they are just getting on with it. Developing is much more rewarding that prattling on about it ... so.... click
Thursday, March 08, 2012 1:33:47 PM UTC
I have to admit, I was basically a dark matter developer probably up until recently, but this year Ive been having quite the self discovery. A part of this is discovering the absolute wealth of knowledge that Twitter can provide, which basically just leads you to everyone's blogs which are incredible.

Personally I have found it to be a really hard time balancing the sheer amount of volume of information Ive been discovering against actual time I have to consume everything. Then I get into a mental funk where I hate myself for "getting on the bus" so late (well more so wasting the last 4+ years NOT consuming this information as much as I am now).

Another part of the problem is (many) years ago I was an avid classic asp developer, and I had such a hard time adapting to asp.Net in general - a part of this was my job, which didn't really make it any easier, and the other was just the abrupt change of directions .net made versus classic asp. Struggling with that for years before I really got comfortable with 2.0, only to realize that 3.5 was out, 4.0 in beta, and by the time I started messing with 4.0, 4.5 was in beta!!

That being said, I truly believe everything happens for a reason, and I have been having a blast learning all of this new and exciting technology. Its just a constant battle to NOT let oneself get completely overwhelmed and just walk away.

As always, thank you for running such a great blog and being a part of the amazing Microsoft community of developers!

Oh, and thank GOD for Pluralsight!!!!
Jason Krol
Thursday, March 08, 2012 2:24:25 PM UTC
I don't consider myself a "dark matter developer" or a "jobs worth". But I do spend my working life on products that were obsolete before some of your readers were born. For older developers, that is both a skill (squeezing the last drop of power out of products that were never designed to be used that way), and a necessity (any organisation with tens of thousands of employees cannot afford to replace all its core systems and upgrade its infrastructure every time a new product hits the shelves). We take pride in implementing cutting edge innovative solutions on trailing edge machines. In that way I help maintain hardware and software investments worth millions of dollars built up over decades. For a new company, using the latest toys may be a good thing; but for those of us in the real world, it is a luxury that cannot be objectively justified.

I still read blogs, follow trends, learn new languages etc., but I have been in the business long enough to be able to separate the wheat from the chaff; taking on board the good bits; and applying it in legacy environments.

It should not be an 'either / or'; but a 'best of both'.
JSC42
Thursday, March 08, 2012 3:01:00 PM UTC
**
My coworker Damian Edwards and I hypothesize that there is another kind of developer than the ones we meet all the time. We call them Dark Matter Developers. They don't read a lot of blogs, they never write blogs, they don't go to user groups, they don't tweet or facebook, and you don't often see them at large conferences. Where are these dark matter developers online?
**

Not theory or hypothesis - it's fact! These are also known as nomad developers. I've lost count of the times I've polled a group of developers to ask them what blogs they read, magazines they subscribe to, etc, etc. The VAST majority of developers are not engaged. For most that program (I don't say "our business" because our business is one of passion and profession) are not engaged. Whether they get more stuff done or not - I don't know. While there may be some correlation, I don't believe there is any sort of causation here.

The simple fact is this - there are a good number of folks who sling code as a job and they don't give a lick about staying current, learning new things, sharpening the saw, etc. For us - that is completely foreign. There are also a fair number of folks who stick to themselves - but do keep current, look to learn new things, etc.

As for me, as I get older and have been in the business a while, I feel the need more than ever to try and stay at least a little behind the bleeding edge.


Thursday, March 08, 2012 3:30:01 PM UTC
Whilst i think we all understand what your article is saying. The one point that it fails to get across is how much of this is self fulfilling and promoted by people such as yourselves and the companies you work for. Not saying that to be insulting, but the way that products are promoted, positioned and in some cases more importantly dropped / sidelined has in some cases led to a potential lack of trust.

Without going over old ground the good old silverlight debate is a perfect example.

"persuade you company to send you yoi Tech-ed / Mix come and meet the industry experts etc"

2 years on Where are they now ??? raving about Jquery / HTML5 / Node.js / Working in Disney / playing with Netduino !

There is a striking parallel here with the behaviour of software evangelists and politicians. Whatever is the next best thing i should be talking about this week.

The rest of the world often has a job to do and learning newer more complicated stuff is a full time job in itself.

A poor but relevant metaphor might be if you and a number of other well known evangelists were to form an orchestra. Each pick an instrument and learn it to a standard you can perfrom in public then when you are all set to go on stage a week before you do just swap instruments !!!

don't get me wrong I love technoology and the pace of change but those of use that invest our time in trying to stay current with everything day in day out might want to step back and reflect on somehting mrs M said the other day.

"You might enjoy working on that computer most of the time, but that kinect and your comapny won't miss you in 5 years from now when it's moved on and obsolete, but your kids will be grown up and gone and what in the end have you actually achieved?"

food for thought !

B.....
Thursday, March 08, 2012 3:34:47 PM UTC
nice one. well then you should mention Black Matter managers too and black matter BA and whole Black Matter SDLC :) it's a whole cummunity.
Ion
Thursday, March 08, 2012 5:47:31 PM UTC
@Ion - funny, but sooooo true. i've been fortunate to have a secure state position for a while, but they move sooooo slow and have no interest in being progressive. as such the developers either grow complacent, or unsatisfied.

@Json Krol - +1 for the Pluralsight plug; I'm really liking their post-course "assessments" - though I'm not sure how much a considering employer would respect them. Tekpub and Peepcode are awesome too if you haven't checked them out.
Thursday, March 08, 2012 6:28:03 PM UTC
I think Scott has a 29 hour day, I could not do all that and still eat and sleep and spend 8+ hours at my F/T job desk, so kudos!! I lurk and read, try to stay somewhat current (MVC3 EF 4.1) and just reading the comments I saw 2-3 technologies/methodolgies/etc that have me saying "crap, gotta go Bing that term."
One page web, sounds like Classic ASP. MVC3 EF creates multiple pages for you, just right click and create... Guess I could go in and make some of them partials...
That is the kind of thinking those of us that belong to the DMD Army do daily (at lunch when I am reading Hanselman or Gu, or xyz).
Thursday, March 08, 2012 8:59:21 PM UTC
The tone of the article is both condescending and patronising, whether intended or not. It is also presumptuous in the sense that it assumes that your domain is the extent of all significant development.
You characterise yourself, and others like you, as 'pushing things forward', 'pushing the limits', 'pushing towards the next big thing'.
You claim to like 'working on new stuff and trying new ways to solve problems'.
You then go on to characterise others as being the type of developers who like 'chilling and punching out at 5.01pm', 'using tech from ten years ago' and refer to ASP,NET 1.1, VB6 and PHP.
I have worked in the games industry and I think that those I worked with would consider their efforts to push hardware to its absolute limit as far more cutting edge than tooling around with the latest web api or scripting language. Just a guess.
I would consider myself dark matter developer by your basic definition. No blog, don't tweet. no facebook account. Outside of my immediate environment you wouldn't know I exist. Though I do appear to read blog posts...
I currently work with a company which, among other things, explores massively parallel architectures (e.g. gpu programming targeting systems with 16k+ gpu cores), vertualisation, interpreters, JIT compilers etc. Some of which forms the basis of the toys you might play with, possibly even the ones your latest greatest tech is utilizing. Technology that might be categorized as bleeding edge, but no one at the company writes a blog, or tweets, or posts facebook status updates, or even attends large conferences (well, maybe the biz dev guys, they have my sympathy).
My language of choice is C with some assembly thrown in and occasionally C++ when pushed. Archaic stuff, I must be close to fossilisation.
Guess some people just think that its not what you use but what you do that defines being progressive, and that the latest new thing isn't necessarily synonymous with progress. It may also be true that those that aren't shouting may not be drudging either.
It appears that two is an impossible number, even when it comes to characterising developers. Disappointing article.
Thursday, March 08, 2012 9:27:57 PM UTC
Snark Kent does have something of a point. I think Scott and some of the other bloggers do a great job keeping interested "dark matter" programmers informed. It would be nice if the Original Post also added some other scenarios such as the sort of thing Snark Kent mentioned, as well as others, for example folks doing some very cutting edge, but not-in-the-spotlight software engineering in the semiconductor industry in order to push the bleeding edge for all of those things that make "wired" life possible for everybody else. Programming isn't all customer-facing web stuff, and, in fact, the lion's share of the value-added in software comes from systems that almost no one will ever encounter in the wild. I'm sure there are myriad other examples of truly valuable, kick-ass, cutting-edge work being done under the radar of the "ain't I God's gift" web or iOS programmer.

I don't at all categorize Scott or Atwood or Joel Spolsky this way; it's really more a function of the many commenters who seem often to do the equivalent of deciding that because a civil engineer (let's say) is not an obsessive reader of Backhoes Today or Girder Week and Crane Technology--and never invented a new kind of hammer--he must not be working on anything important or cool.
Alt Numlock
Thursday, March 08, 2012 10:02:05 PM UTC
All of the 'dark matter' developers I know choose to never publicise their techniques because they make their living by keeping them confidential, or are legally bound to do so!
You are asking them to (at least) give away their bread-and-butter, or even to risk prosecution.
Michael Kingsford Gray
Friday, March 09, 2012 2:49:20 AM UTC
I think you can get stuck in the dark matter 'cycle'. I spent the last 6 years creating frameworks and customising nettier template and creating code snippets so I could be more comfortable and productive in my full time job. I got sick of supporting all the stuff I had written while not being given opportunities to do new stuff. So I left. Now doing contract work, loving it, and moving out of the dark matter, learning and thinking about how I can do things better and be more productive with the new technologies.

Remember though that something stay the same like, good OOD (I use TDD to help drive this), understanding what people want ( :-P ), trying not to be an arrogant arse hat etc.

I think we all go in cycles as required. So here I am reading something by this Hansel guy.
SleepyBoBos
Friday, March 09, 2012 4:11:01 AM UTC
Snark and Bruce - Thanks for your excellent critical comments. You both make excellent points. It was not my intent to be condescending. I was "getting things done" for many years before I started this gig. While I'm not an evangelist, it may seem like I am one with all this blogging and what not. I spend more time trying to better serve the customer and affect the direction of the the ridiculously slow-to-steer ship that is Microsoft than blogging (as I do that at night) but your points are well taken. It was also not my intent (another failure it seems) to imply what we're working on is the sum total of innovation out there - it's certainly not.

My goal was simply to juxtapose the Loud with the Quiet to perhaps remind my Loud Friends that the world's work is being done by a largely underserved Quiet majority.
Friday, March 09, 2012 4:41:44 AM UTC
I had a chance to work with some Dark Matter developers last year, they often worked with legacy VB6 and VC++ 6.0 code. I also worked with Thompson Toolkit, that I didn't know about before, I later found out that they was a build toolkit from 1996 :).

They don't need to read blogs, follow people on Twitter because none of these people talk about what they are using.

Surely they also don't have any reason to know about fancy things that we are working on for now such as MVC3, jQuery, NoSQL, cloud etc. since their product is a 20 year old legacy enterprise application. Even on its own forum, you can see only a few new post every week.
Friday, March 09, 2012 2:00:50 PM UTC
I'm probably one of the 99%, and while I understand your point, I don't really think it's accurate to assume we only use old outdated technologies because that's what we're comfortable with. The work I do now is with MVC3 and all the latest web techs that go along with that. I'm good at what I do and I know it. I don't feel the need to pound my chest so people can see what I can do. I'll work at home after hours, I read blogs and listen to podcasts. I educate myself with the latest techs. However, I have a family that I enjoy spending time with and some hobbies that I enjoy which have nothing to do with technology.

So while the 1%ers are blogging about how they just wrote a Phonegap app to talk to a REST service running under Ruby, I have real work to do and a schedule to keep.
Friday, March 09, 2012 2:08:23 PM UTC
Alert CERN, call CNN, stop the presses (newspapers are still they way most people get their news, right?)I am dark matter.

It's not that I'm in the dark, I know what's going on around me. I'm in tune with my surroundings. I read blogs, read about the latest/greatest tool, I know what Stack Overflow is, try to stay informed about where and how my industry is moving; however, I just don't feel the need, or desire, to shout from on high about what I know and why I think you should do what I'm doing. It's just not my style. I prefer to be in the background doing my thing without fanfare.

My units motto in the military is "The Quite Professionals." I've carried that over to my civilian career and I like it that way.

The bottom line is that we all just need to get things done. I'm not suggesting that the people who do tweet and blog don't. It's your personality and you do it because you love doing it. And honestly, you're very much needed and appreciated.

I'm grateful for all you folks that do blog. That's how I stay informed while quietly going about my day doing what I love to do.

So thank you to all you visible matter folks out there colliding and creating the energy needed to keep our craft going.
Mark
Friday, March 09, 2012 2:35:25 PM UTC
Go with peace.
I don't program on ASP in Mexico.
I program on C++ in xXXXXx.
I program embedded devices, which create your stupid cloud.
I personally don't participate in "the cloud" because giving up
your independence is stupid squared.
I don't read blogs, because in the last 10 years there was nothing worth reading.
darkD
Friday, March 09, 2012 2:41:24 PM UTC
Great article, great comments! I work as a software consultant, helping real companies and government departments. I attend some conferences and local IT get-togethers. Over the years I've done a decent amount of open source. Like many others commenting here, I consume blogs and articles and presentations voraciously. I try out new stuff at home.

But when all is said and done, one month I might be helping a client integrate Oracle Forms on a J2EE 1.4 app server with IBM FileNet, the next month doing custom extensions for an existing VB6 app that some customer has, the month after that helping fix a Struts application, and the month after looking at some C# 2.0 code.

In my spare time I play with Scala or Haskell or F# or Clojure or Ruby or C# 4+ or C++11x, but that's just to keep my brain limber. The odds of encountering a client that uses any of this are tiny. More productively I keep abreast of software engineering thought: lean architecture and DCI and modern testing ideas and the latest approaches to requirements gathering. And that I do try out with clients. But I don't really have time to blog about it.

If you develop long enough I think you arrive at a realization that it's solid software engineering principles that matter, and that those don't change rapidly. I think it's cool that a 1% or 5% constantly experiment with tools. I won't deny that JSF or ASP.NET MVC, for example, have made my life easier; but I also remind myself that some years before either of those matured I worked for a company where we churned out solid web apps using ColdFusion. I don't think I've ever run across a programming situation, not once in over 30 years, where you couldn't adequately solve an IT problem with solid non-bleeding-edge tools available at the time.

A corollary to the theme of this article is the relevance of what a lot of the cutting-edge 1% or 5% of developers are doing. Cloud and mobile and NoSQL and all that good stuff is fine and wonderful, but we haven't even come close to solving the basic SDLC loop yet: requirements, design, coding, test, maintenance, in whatever particular methodology you care to employ. The majority of projects still have a hard time completing good solutions on time and within budget. Are all these tools and new infrastructures going to solve that somehow? No. And it's that problem that the "Dark Matter" developers actually deal with - every day.
Friday, March 09, 2012 3:17:01 PM UTC
Great post about the devs out there keeping the lights on at organizations across the world. While I agree that a lot of the older technologies are powering the world because they are proven with plenty of documentation available, I feel that we still need to prod the "Dark matter" developers. I recently (2012) saw a development team get laid off because they did not retool from VB6 to .NET. While it's probably unnecessary to jump on the latest bandwagon for every project, sticking with what you know will lead to a dead-end one day.
Friday, March 09, 2012 3:25:16 PM UTC
Here we go. It was 501 developer, now it is Dark Matter Developer.
Please don't tell me that it is not insulting, well-meant and purely theoretical. You are rare case of the speaker guy who actually can have shit done - quick and solid. But for you there are 100 of speaker guys who had no idea how to code another list box, when they have to. And they are all with you shoulder-to-shoulder pointing their finger at those Dark Matter people with derision.
Saying something like that for person who holds a unique position is an ethical equivalent of those who are CEO referring to those who are not as losers. 501 developer works 8 hours to gain his fixed wage.
Michael
Friday, March 09, 2012 4:45:54 PM UTC
Thats all very well put, but Dark Matter still only constitutes 23% of the whole universe. Dark Energy developers are the ones you have to find with a whopping 74% ;)
Friday, March 09, 2012 8:30:13 PM UTC
"...they are just producing. (Or they are just totally chilling and punching out at 5:01pm..."

The nice thing about being a dark programmer is that these things aren't mutually exclusive.

Just sayin ;)
Saturday, March 10, 2012 3:19:37 AM UTC
I think it is important to remember as well that not everyone who is loud is really "pushing things forward". There's a million people banging away at keyboards and blogs and githubs right now, but how much of that is actually value-adding work?

Almost all of the awe-inspiring developers I met who changed my own understanding of programming and professionalism as well had no blogs - barely even an internet presence. Why? I can't be 100% sure, but they always seemed more interested in producing cool stuff than in exchanging ideas.

That being said, evangelism is no doubt important for the lay developer, but I would also divide the 1% pushing things forward in half. One half likes talking about it, the other half doesn't. Both halves are needed to push things forward.
Eric
Saturday, March 10, 2012 11:01:18 PM UTC
It is also important to remember that not everybody is part of your community. It is not like "the 1%" is one group and everybody else is another.

I spoke with Scott Hanselman once and at one point he said "Maybe I will see you at a Code Camp". "What is a Code Camp?", I asked. Scott's response? "Are you a developer?". We had already discussed that I was. Clearly, it was difficult for him to understand how that was possible if I had to ask this question.

It is not exactly as if I was using VB6 in the third world either. For one thing, I had listened to most of Hanselminutes by then. In addition, I had just recently written a .NET compiler for fun, moved a client to distributed version control, completed a sophisticated business app, modified Mono so that I could use <%: %> in ASP.NET 3.5 on Linux, and achieved a rep of over 2000 on Stackoverflow (not much I realize, and now higher, but enough to not be punching out at 5:01). I saw Scott speak at Dev Days in Seattle a couple of weeks later but have still never been to a Code Camp.

My point is not to harsh out on Scott. I admire him greatly and this is another good post. Rather, my point is that even when we know we are in "the 1%" we might be in "the 99%" from another perspective.
Sunday, March 11, 2012 12:27:28 PM UTC
Everybody seems to has forgotten mainframes.
Mainframes had black screens reminding of Dark matter.
But there were so little bugs and panic technical situations.
However code was sequential, 100,000+ lines of code at times in a single program. C++ is the basis for modern languages. Newer challenges are created for newer programmers and architects.
Sunday, March 11, 2012 12:46:03 PM UTC
One morning in 2004 I took the train to work, and sat down next to this young man who was looking at a blue binder filled with continuous feed green bar paper complete with tractor holes on the side. I looked on the page, and it was covered with COBOL code in those fuzzy letters indicative of the old IBM line printers like I used in college in the early 1980s. I asked him about it, and he said it was code for the back end of a gas station point of sale system. He explained that some bug had cropped up during the night, and he was on his way in to the office to work on it.

What struck me was that he was one of the most cheerful people I had encountered on the train, and he exuded enthusiasm for his work as he explained it to me. He saw his work as vitally important for the success of his business and for his customers. The rest of the trip, he examined his listing with engagement of an avid crossword fan working on the latest puzzle from the New York Times. That encounter left me with a new appreciation for those who are not working on the cutting edge of technology.
Sunday, March 11, 2012 6:49:26 PM UTC
Great post, loved the comments!

I think we have 200 devs in the company I am working for, but never met any of them online.
Still the ones I know are smart motivated engineers that read books, blogs and produce high quality code solving quite complex problems.

Most web applications that we write are in asp.net mvc using latest versions of the ms tech stack.

Our desktop applications are worse off. While WinRT is looking mighty fine, most of our customers will not switch to windows 8 on release. I guess 30% of them are still on XP. So the need to stay current here is much less. It's even painful looking at all the new goodies we cannot use since we still have to PInvoke or use COM. grrrr.

I really like bloggers and speakers and thank you very much for channel9. It's just that our customers have problems were technology is just a small part of the solution.
SentientNr6
Sunday, March 11, 2012 6:51:48 PM UTC
Hypothesize? I am sure of it. For the first part of my career I worked in these kind of shops both small and big. Most of the folks would come to work, write code and go home. There was no interest or even awareness of what was going on outside. There were usually one or two folks in the org who really had any idea of emmergine practices/technologies

For me I was interested in new ideas, but I was too busy writing code to get out there. I also did not know about the community and the blogosphere. The only time I peaked outside was if I went to a conference like VSLive or read a technology book like POEAA.

I know it has been six years since I joined Microsoft, but I cannot believe it has changed 'that' much.
Glenn Block
Sunday, March 11, 2012 6:52:22 PM UTC
Ooh yeah, please add a like button to the comments section so I don't have to write a comment myself.
I prefer staying in the dark.
Thank you! :)
SentientNr6
Tuesday, March 13, 2012 2:37:28 PM UTC
Forgive me since I did not read EVERY post and this may have been said.

I enjoyed reading this article but it left me with a bit of a distaste because it says we need to reach out to these other folks and get them to be more like us and join the online community. You sort of caught that and mentioned that we could learn from them how to get stuff done. But we have to be careful when we think our way of using technology or interacting is better. Couldn't it be that we all have different roles in the geek world we live in? And that it is ok for each of us to continue doing what we are doing, whatever that is?

As a few people mentioned, I see that there are different types of folks: Developers, Programmers and Coders.

Devs are like you and me: we try things, we read geek stuff at night, we break stuff and then fix it, we are the ones stepping into new territories and then teaching it at conferences/user groups, blogging about it, etc...and we love it!

Programmers work the 9-5 and then go home...this majority group expects us to continue to blog the things they will use 9-5 the next day. They will take our ideas and tweak them into the practical programming they do. They don't go to conferences or user groups unless they are forced to, the company pays for the trip to Vegas or the topic suits them and their own need/want. They like their role as is.

Coders are another minority, working hard, extra hours as needed, plugging away in the older technologies, happy as a clam, or at least resistant to change.

Each group is happy where they are and it is very difficult getting them to change groups. In fact, it is futile and frustrating to expect them to change. It is detrimental to want them to change...we need idea people, we need technical people to take the ideas and bring them to the practical, and we need the workhorses to bang out code for as long as we need to support the codebase.

Each has their role and we are better served letting each group do what they do, finding the right place for them in our companies (where admittedly some groups won't fit) and putting good leaders over them to help them be the best they can be in their role.

Keep doing what you are doing, since you serve a minority but large group of true developers very well. Thank you! (On a side note I would love to see a post or two on your success in balancing solid technical community involvement with family and personal time. It is inspiring to see such commitment to both sides of life, and for it to work so well. Idea-oriented, forward-looking guys tend to not be so balanced.)
Greg Howe
Tuesday, March 13, 2012 8:31:09 PM UTC
Hi,

I haven't read the entire comment list but I have some $.02 about the company I currently work with...
We work on a software product which makes use of Datastage TX to handle transforming text from one format into another - we need to process many of these plain text messages in various formats and route them to the appropriate destinatiosn. The people that write the DataStage TX 'code' to transform the text are considered 'Developers'.

Writing 'code' in Datastage is not like writing code in Java/C#/C/C++/Perl/PHP/Python, etc. I guess something somewhat relatable though is 'Developers' at my job look at specifications of the input to expect and make sure they have correct output, again to a certain specification.

Backtrack to when I interviewed for this company I was considered because I have a technical background from school (Comp Sci degree with knowledge of Java) so I would learn the system from a systems analyst point of view then transition to a developer. Little did I know a developer meaent what I described in the paragraph above, not what you guys think of as a software developer who writes code, unit tests it, collaborates with others on problems, code reviews, etc.

I guess my issue is, what qualifies you to be a developer?.

I don't think any of these guys follow the tech industry at all or understand a software stack (from data layer up to the application layer, for instance) like a traditional software developer. These guys may be "Dark Matter Developers", that to be quite frank, would not contribute too much to the online developer community. Maybe I've got a jaded view because I feel misled when I got hired for this job.
Wednesday, March 14, 2012 1:54:52 AM UTC
(Typed by a "getter-done"-er on Window 8 CP via touch)

Can you see me now? ☺ No? How about now? Still no? Got a 404, right? Wait, how about now?

Now that wasn't very productive, was it?

Dark matter devs would be a good developers life podcast.
davewill
Wednesday, March 14, 2012 8:43:11 PM UTC
I think that should probably read "renamed it three times and then deprecated an old one just while I was typing this sentence." :-)

Thursday, March 15, 2012 11:29:27 PM UTC
I like working with devs who care about what they do but some care a bit too much. They obsess over every trchnology choice or endlessly evangelise new tech that is not relavent to our ecosystem. Often what they fail to realise is that the people, process and team are more important than the technology choices for ensuring successful delivery of projects and products.
Friday, March 16, 2012 3:25:05 AM UTC
On one of the comments above, XP is still used extensively on the Enterprise and most enterprises dont see a reason to move to Win8 or Win7 any time soon. It just works. Right now im developing on both really old school, cuz its needed and on really new stuff. From Cobol code for business software, web services to access Databases and interface with this old applications, ASP.net to manage new challenges, Java application on the Web Front End. It really is all over the place.
The point is this, most off us dont get to use all this new stuff cuz the companies that we work for are not ready to use them, we try to push the envelope for the to use new technologies but like all things in mature companies and organizations, change is slow.

Keep the job perfecting new technologies for us, we a looking and following blogs, and post and all that. We just also have real life with kids, schools, football and all the other stuff that make life worth while, you'll get there one day also and i hope that you have made a dent on moving tech forward.
Eddie Zambrana
Thursday, March 22, 2012 7:19:08 PM UTC
Scott,

I found this particular post of yours extremely fascinating. While we're categorizing programmers, I'd like to add my category to the list-- fringe programmers. Maybe we don't have time, resources, or interest to focus on programming as a main priority in our lives and we're ok with that.

Personally, I really enjoy reading your blog because you do give examples from all kinds of different development areas and with a host of different languages. I read articles and blogs about programming as a way to informally school myself at a slow slow slow pace, and I create small application centric scripts once in a while. Learning better logic and looking at more professional code regularly helps me do that to the best of my ability.

I am fascinated that you say you can identify some parts of the seemingly ever changing tech environment that are here to stay. Would you consider doing a blog post on those things that you predict will be around for a while and why? Any of your other readers who are on the fringes (your posts are extremely accessible because you are effectively giving us expertly commented code, in my opinion) like me would probably also appreciate it.

Thank you for the fantastic content.
Friday, March 23, 2012 8:48:59 PM UTC
Dark Matter coders just don't use old tools to code. I think their main characteristic is to be so busy that they don't care about going online. Or maybe they just go online to find the piece of code they need.

On the other hand, I love old software: Win XP, VS2008, VB6, notepad for C++... those are my tools :D
Tuesday, April 03, 2012 9:04:11 PM UTC
Interesting read.

Thanks all - from a young developer trying to find his place in the craziness that is the software game.

It's about doing what you get the most out of. Finding what that is is the non-trivial part.
Wednesday, May 16, 2012 1:03:22 PM UTC
It's pretty strange to realize myself as Dark Matter :D

When you maintain an old project based on old technologies you haven't enough time to be online and test all new technologies and features, play with the latest beta etc. Just no time for pushing limits.
Monday, July 02, 2012 8:14:12 PM UTC
Maybe it's just me but I find this very sad. As a developer who loves to spend time reading blogs and learn, it's a bit disheartening working in places filled with these "dark matter" developers that don't care about self improvement. Often they take this so far to not even care about things like quality or design patterns or anything other than hacking out code like a monkey.

It makes me sad that so many shops are infested with these types of developers and just wallow in mediocrity.
Wayne Molina
Sunday, July 15, 2012 10:35:59 AM UTC
Dark matter can be seen, you just need to know where to look, same with the dark developer.

http://www.cosmosmagazine.com/news/5765/scientists-shine-light-dark-matter
Monday, July 16, 2012 3:41:56 AM UTC
Give me a Windows 7 box 50 years from now with VS 2010 and C# .NET 4.0, and I bet it will still be able to do some crazy-ass shit! I'd enjoy becoming a dark matter developer with these technologies. I'll never shun new ones and I do enjoy reading blogs, but having spent the last year tirelessly working the kinks out of a new technology, being more productive and taking less risk by using proven technologies has become much more attractive. Great post Scott and thanks for the interesting read.
Matt
Tuesday, July 17, 2012 8:39:40 AM UTC
I tend to find that these Dark Matter Developers are more consumers of information than providers. And there's absolutely nothing wrong with that.
Some people prefer to see what's going on around them without actually getting involved.

To be honest, it's actually quite difficult to participate online (in my experience) as you're not quite sure which blog/twitter user etc. to follow, let alone comment on, participate in etc.
And I do think the biggest issue here is, as you said, the fact that the bloggers out there are focused on the latest and greatest technology whilst developers are in the middle of their projects using their current stacks, so they wouldn't have anything to comment on.

It's quite intimidating actually, as you could feel so left behind. And I'm sure most devs out there get a bit despondent when a new release comes out especially as they've just managed to get into the previously 'new release'.

It's tricky and actually takes a lot of effort to stay up-to-date, and as the one comment said, a lot of developers just view this as a job.


Tuesday, August 28, 2012 9:01:19 PM UTC
<-- Dark Matter Developer

This post inspired me to get more involved in the developer community. It took me a few months to lessen my contractual workload, but I've finally started a blog (shameless plug: http://www.adamelmore.net/) and I'm on the path towards change. I've always read technology blogs (this being my favorite), but other than that I think I mostly fit the bill.

Thanks for the acknowledgement, Hanselman.
Tuesday, September 11, 2012 5:40:56 AM UTC
I've thought about this as well but never had a neat term for it. Here's the crazy thing, one could assume that a dark matter programmer is just someone who isn't very passionate, treats it as just a job, etc. But that generalization isn't always true, in fact most of the "dark matter" developers I know are very talented, and passionate they just keep to themselves and value privacy a lot more.

My experience is much the same, I see them "getting things done" and they're really great developers, just not the sharing type like me.
Wednesday, September 12, 2012 8:38:04 AM UTC
I'm a User Group Manager in what many consider to be an old language: ColdFusion. I spoke on CF on a .NET Rocks Podcast last year.

In a non startup-bubble market, in a unpopular language, we manage to have monthly meetings. How is this possible? There's a lot of CF in the 99%. When I try to force a meeting on "modern" techniques (MVC frameworks, new JS frameworks, etc) I get some resistance or complaint. I love new technology but must meet the needs of my community.

tl;dr Better to "old" language that works or cutting edge that isn't even 1.0?
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.