Scott Hanselman

Your Opinion Matters - Screencast Techniques Survey RESULTS

January 15, '08 Comments [17] Posted in ASP.NET | Microsoft | Musings | Screencasts
Sponsored By

One month ago I asked you, Dear Reader, to participate in a survey about Screencasts in order to help me, and indirectly Microsoft, improve our screencast techniques. I got 1000 responses in just a few hours and I thank you for it!

You also wrote a lot in the free form comments field with 319 of the 1000 responses including typed feedback. These comments were read by not only myself, but (surprisingly) my boss and ScottGu, so that's cool. It's also cool that 1/3 of you felt compelled to offer more feedback. Thanks!

Here's the feedback we received.

Answer Options Strongly Disagree Disagree Neutral Agree Strongly Agree Rating Average Response Count
Seeing a PIP (Picture in Picture) of the presenter is more engaging than no video at all 38 147 286 394 135 3.441 1000
Panning and Zooming to key areas enhances the experience 15 50 194 500 241 3.902 1000
Screencasts should include high quality audio 5 36 138 436 385 4.16 1000
Mouse clicks should be called out visibly 24 95 300 439 142 3.58 1000
I wish I could watch a screencast on my portable device 54 131 359 253 203 3.42 1000
Screencasts should have minimal visual clutter 5 18 157 531 289 4.081 1000
Too much PIP (Picture in Picture) video of the presenter is distracting 37 136 249 440 138 3.506 1000
Screencasts should only include audio and a full-screen capture 89 389 308 165 49 2.696 1000

It's hard to see the patterns, of course, as it's tabular data, but here's how I interpret it and a few other ways to think about the results.


results

UPDATE: Here's a much better visualization above from Dylan Beattie. Thanks Dylan!


Screencast_Survey_Results

UPDATE: Here's another visualization from Brian Boatright. Thanks Brian!


Analysis

First, if someone says Agree or Strongly Agree, let's call that AGREE.

Ah, now this is more useful. Looks like folks really agree on these points.

  • 82% agree - Screencasts should include high quality audio
  • 82% agree - Screencasts should have minimal visual clutter
  • 74% agree - Panning and Zooming to key areas enhances the experience

Which is expected. Screencasts should sound good, only show you what you need to see, and some amount of pan/zoom (which is arguably an extension of the "minimal visual clutter" rule) is useful.

Now these are more on the fence, but I did include 1000 results so I consider them fairly valid.

  • 58% agree - Too much PIP (Picture in Picture) video of the presenter is distracting
  • 58% agree - Mouse clicks should be called out visibly

Seems that two much of a good thing (i.e. my (or anyone's) fat head) is distracting. This was confirmed in the comments. Basically, if you're showing something within the interface, or if you're just typing, don't show your head/PIP.

As for the mouse clicks issue, this is a tough one. I think I'll keep the effect, but I'll speed it up and make it smaller, so that we can find a "best of both worlds" option to please the most people.

These were interesting to me. Basically half of folks think that PIP is interesting, but it's clear that it's not necessary. According to the comments those that like PIP feel it "helps one engage with the presenter" and "keeps my attention." Others said "it seemed like you were talking to me" and "I'd be more likely to watch a screencast all the way through if it had PIP at some points."

  • 53% agree - Seeing a PIP (Picture in Picture) of the presenter is more engaging than no video at all
  • 46% agree - I wish I could watch a screencast on my portable device
  • 21% agree - Screencasts should only include audio and a full-screen capture

I was surprised that 46% of people wanted to watch on their portable device. Depending on your point of view this may seem like a small number or a large one. However, it's a significant number of people, in my opinion, if over 460 folks out of 1000 would like to watch a technical screencast on their portable device. This requires less effort on the part of the publisher than you'd think. I'll post about this issue later and what you can do as a creator of screencasts to make them usable on Zunes and iPods and other PMPs.

Finally, 1/5 of folks feel that screencasts (perhaps they are screencast purists?) should include only audio and the full screeen.

Also, as with most surveys, this one was arguably biased towards a specific point of view, namely, mine. One could say it was poorly written and two people did in fact say just that. However, I'm not hiding the fact that I believe judicious and reasonable use of these small techniques can make for a better screencast, but I'll be more careful with future surveys to include both sides of the perspective.

Written Comments

As with my blog, it was with this survey. The interesting content is in the comments. There were 319 comments, but here's a slice:

  • "Would like transcripts of screen casts so that the material can be searched." and "For foreign users who do not speak English as a native language, it is easier to understand the screencast, if there are subtitles in English"
    • This was a common request, as were subtitles for the deaf and folks for whom English was not their first language. I'm looking into ways to do this without breaking the bank. (There's less money for these than you think.)
  • "I'd also like for a lot of the screencasts to be fairly short, so that I can watch them without investing too much time at once on them"
    • Agreed. I talk a lot and I personally like the 45 minute format. However, I think 8 to 15 minute might be a better length.
  • "why not a close up of the text being typed" or "use keyjedi"
    • These were interesting, and I'll look into it. I agree that it's hard to see what's happening with hotkeys and quick cuts.
  • "I really like to see presenter one side, content on the other, clear separation"
    • Interesting idea. I'll try it.
  • "I really think the content is key. If the topic and execution are solid, the presentational stuff is only window dressing" and "Script work and rehearsal go a long way."
    • I agree 100%.
  • "I think Microsoft Webcasts are of poor quality and should be replaced with screen casts. Often issues arrise during a web cast and it is much more transparent to us if they are produced instead of thrown together. No Web Casts more SCREEN CASTS!"
    • Interesting perspective. I see your point, but sometimes Live is better.
  • "Proper support of standard codecs." and "I want to watch this on my Xbox360."
    • I totally agree. If we do an MP4 version, it should work everywhere.
  • "Too much panning and zooming will make me sea sick. Your MVP screencast is very good but it borders on the unbearable. Slower pans are better, one axis only pans are better. Avoid pans & zooms at the same time and get yourself a cinema book for beginners (seriously)."
    • This was one of two comments where the reader was ill from watching the screencast. I think the one-axis pan idea is really interesting and I'll try it out.
  • "Please support wide-screen format."
    • I was very surprised but out of 1000 folks, there was only ONE comment that I use Widescreen (16:9) ratio. I'd have expected a lot more.

Also, I received this one: "there... i filled out your survey... motherf******! ( lets see how anonymous this really is) :)" Note that I've alerted your mom and your boss about your naughty language, Mr. Anonymous. ;)

And finally, "No beards! Beards are distracting!" Hm. I don't think you want to see me without a beard, I look 12. Seriously. More importantly, what are we going to do about this guy?

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Tuesday, January 15, 2008 5:24:16 AM UTC
I think that table is calling out for some Excel 2007 Conditional Formatting love.
Tuesday, January 15, 2008 7:38:20 AM UTC
@Brian, in my days we used to call this CSS ;)

@Scott - Whatever comes out of this survey, I think that the no. 1 outcome is "Please do make as many pod/screencasts as you possibly can. The crowd just LOVE'em"

and keep up the good work, blah blah.

Also, keeping a beard for the sake of not looking 12 is a solid approach, one I take myself ...

Tuesday, January 15, 2008 8:47:22 AM UTC
I just loved your PIP presentation. Communication is a high percent non verbal, so seeing your expression while you were presenting was good for me. But even if you decide against PIP presentations, I would not mind two separate videos. I could just run them in the same time.

Now for some weird Siderite ideas:
Isn't there some sound/video time synchronisation feature in Silverlight or something? It would be nice to have a variable number of videos that would be able to run in the same time. People could stop some of them to preserve bandwith and such or choose to see them all. Same sound file, more mute video files.

And one more thing: is there a video compression algorithm that would actually preserve the text written? It's terribly annoying to see someone write code and not be able to understand what he is writing because everything is blurry.
Tuesday, January 15, 2008 1:07:45 PM UTC
I wanted very much to do the original survey to express the opinion that the audio quality should be paramount. I hear too many 'casts where the presenter is right on the mike, but the guest is not.

Balance the audio!! It's not helpful for the listener to have to ride the volume control to hear the one and not be deafened by the other.
Tuesday, January 15, 2008 1:55:42 PM UTC
Hey Now Scott,
Some of the comments are funny like the beard & anonymous one. Screen cast are great way to learn I feel. I watched a few more last night. I agree with the one comment how screen casts are better than web casts but I really enjoy the webcast too in a different way, I like the live presentation. A mix is important.
Hanselminutes fan,
Catto
Tuesday, January 15, 2008 4:21:03 PM UTC
I've not been a huge fan of podcasts, screencasts, et al. The main reason being one of time. When I am at work, where I have a computer and connection to the net, I don't have time to watch 5-10 hours of casts each week or even in a month. When I am at home I am home and try to keep my time on the computer limited. If I am buried in my 'puter then I am not being a husband and father. So, the time left is my commute. And I rarely carry a computer back and forth anymore. So, this leaves me with my bright and shiny Samsung i760 WM6 device. I spend 25 minutes in the morning and 25 minutes in the evening on a bus and would love to use this time to watch and listen to casts.

I like the casts that show off the new stuff. I'd like to see more vignettes, 2-way dialog, on *when* to use the new stuff. More of a round table approach using a blog as the feedback mechanism to help focus the next vignette. Hope that makes sense. :)
Tuesday, January 15, 2008 4:33:32 PM UTC

Personally, I don't need PIP. What if the PIP area is obscuring an area of the screen I need to see?

I guess to make everyone happy, offer two versions; one with and one without. I am assuming you're adding the PIP after the fact so you're not really doing extra work.
Abdu
Tuesday, January 15, 2008 4:43:36 PM UTC
This is an insightful survey, Scott. Thanks for doing it.

Interestingly, the feedback we receive related to our MIX and PDC session recordings is that visitors like the "talking head." It costs us extra to include this feature, and based on what we've heard to-date, we think it's worth doing. I'd be curious if the "picture OUT of picture" format we use at http://sessions.visitmix.com/, for example, is considered useful by this audience.

I understand that we're not really producing "screencasts," per se, but there is definitely an element of similarity.
Tuesday, January 15, 2008 4:57:57 PM UTC
We're struggling with the same questions on our new video show (no advertising here :) ).
Here are a few questions I'd like to have answers on:
1. Would you like to have the ability to run the screencast at higher speeds (meaning X1.4 - Like media player lets you?)
2. What the best length for a screencast - 5,10,15,30+ min?
3. Would you like to be able to subscribe thru Itunes?

Tuesday, January 15, 2008 6:44:40 PM UTC
It's interesting, if you weight the answer options differently (linear, logarithmic with different intensities) you verify that your top three are important:
1. Screencasts should include high quality audio
2. Screencasts should have minimal visual clutter
3. Panning and Zooming to key areas enhances the experience

But, the more your skew the weighting towards strongly disagree (i.e. avoiding complaints), "I wish I could watch a screencast on my portable device" moves towards 4th place. Otherwise, the more skew the weighting towards strong agree, you simply reaffirm this order:
1. Screencasts should include high quality audio
2. Screencasts should have minimal visual clutter
3. Panning and Zooming to key areas enhances the experience
4. Mouse clicks should be called out visibly
5. Too much PIP (Picture in Picture) video of the presenter is distracting
6. Seeing a PIP (Picture in Picture) of the presenter is more engaging than no video at all
7. I wish I could watch a screencast on my portable device
8. Screencasts should only include audio and a full-screen capture
Tuesday, January 15, 2008 7:14:46 PM UTC
Image quality and audio quality are paramount in a screen cast, i.e. the viewer should be able to both see the area/item of focus and be able to clearly hear the presenter. However, the idea of high quality (audio or video) is somewhat subjective in that it can be construed to mean high bandwidth, which can mean high quality gibberish. On the video front Camtasia makes sure that the image isn't out of focus so that 'quality' aspect of the picture is handled, the panning and zooming helps both focus attention to the relevant portion of the screen and will help with playback on hand-helds. For the audio, a good headset boom microphone that will keep the mic in a fixed position relative to the mouth will go a long way to maintaining a consistent and clear audo; no need for high bandwidth with the human voice, just a waste of bandwidth and enlarged file size; but you knew that.
Steve S
Tuesday, January 15, 2008 8:04:51 PM UTC
I've put together a quick graph & spreadsheet which shows these results a bit more clearly if anyone's interested:

http://www.dylanbeattie.net/hanselminutes/

Interesting reading, though - thanks for sharing these results with us!

Oh yeah, and I also wear a beard so I don't look 12 years old... I wonder how many of us there are? :)


Tuesday, January 15, 2008 10:27:16 PM UTC
I had no idea this survey happened... my suggestion would be, where applicable, have the source code of the project available for download with the screencast.
Eric
Tuesday, January 15, 2008 10:38:23 PM UTC
Can you blog more about WCF and WPF please, Feels like these topics are quite esoteric...

Thanks,
Yash
Wednesday, January 16, 2008 1:38:17 PM UTC
You can always make the screencasts more engaging using foul language. i.e. Dropping a random f-bomb in the middle of a discussion about ICloneable is probably the best way to snap someone back to high levels of attentiveness and anticipation!

Good Luck!
anonymous
Wednesday, January 16, 2008 6:57:19 PM UTC
The other Scott's star rating is up 89% in the last week, probably from your link to his profile.
That might actually give him a better shot at the next after-school special "big brother" role he auditions for.
Good job!

Oh yah. And nice post... I think I have to agree with the keeping them short comment. One thing I've liked about some of Rob Connery's Subsonic web casts is that they are about 10 minutes long... even though I don't use Subsonic, I've found that 10-15 minutes is a great length, if you can pull it off. Less chance of getting interrupted, distracted, or bored.
Thursday, January 17, 2008 5:56:23 AM UTC
Good timing on the survey, well at least for me. I just got Camtasia 5 and spent the weekend going through the tutorials. The PIP feature was one I wouldn't have thought that useful to the viewers, but apparently I was wrong if the survey results are any indication. I guess a short headshot at the beginning and another at the end would be sufficient for a short 10-15 minute video. Any longer and you'd probably want one more somewhere inside.

I can also see the PIP being appropriate when you get to a part of the presentation where you have to do a lot of explaining, or give some background and what is currently on screen is not of paramount importance.

Hey Scott, show idea: Maybe you can interview Carl about DNRTV sometime. I've noticed in DNRTV there's never a headshot, nor do I recall any titles being used. Just curious why Carl makes some of his editing decisions plus hear any tips for production processing he wants to pass along.
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.