Scott Hanselman

Penny Pinching in the Cloud: How to run a two day Virtual Conference for $10

April 26, '13 Comments [28] Posted in Azure | Open Source | SignalR
Sponsored By
DotNetConf Logo

We've just finished Day One of "DotNetConf" our community-run free online conference for developers who love the .NET development platform and open source!

UPDATE: All the Videos from both days of DotNetConf are now online and available for on-demand viewing!

The Conference Platform

It seems funny to call the software our conference runs on a "platform" as that sounds very "enterprisey" and official. In the past we've done aspConf and mvcConf with sponsors who helped pay for things. We used Channel 9 and had a studio and streamed either from Seattle or using Live Meeting.

However, this year we wanted to do it on the cheap and more distributed. We wanted speakers from ALL over in all time zones. How cheap? About USD$10ish we figure. I'll get a complete bill later, but we basically wanted to scale up, do the talks and scale down.

Video Broadcasting and Screen-sharing

  • This year we are using Google Hangouts with their "Hangouts On Air" feature. A "dotnetconf" Google Account invites the presenter to a Hang Out and check the "on air" box before start the hangout. Then we use the Hangout Toolbox to dynamically add on screen graphics and speaker labels. Everyone sets their resolution to 1280x768 and the live stream ends scaling down to 480p.
  • Once you hit "Start Broadcast" you're given a YouTube link to the live stream. When you hit End Broadcast, the resulting video is ready to go on your YouTube page within minutes. The hangout owner (me or Javier) then clicks "Hide in Broadcast" and fades away. You can see I'm faded away in the below screenshot. I'm there, but only if I need to be. When there's only one active presenter the Hangout turns into a full screen affair, which is what we want.
  • Important Note: Rather than an 8 hour Hangout, we started and stopped as each speaker did their talk. This means that our talks are already discrete on the YouTube page. The YouTube videos can have their start and end times trimmed so the start isn't so rough.

Google Hangouts On Air

The Database

Surprise! There is no database. There is no need for one. We're running a two page site using ASP.NET Web Pages written in WebMatrix. It runs in the Azure cloud but since our dataset (speakers, schedule, the video stream location, etc) isn't changing a lot, we put all the data in XML files. It's data, sure, but it's a poor man's database. Why pay for more than we need?

How do we update the "database" during the talk? Get ready to have an opinion. The data is in Dropbox. (Yes, it could have been SkyDrive, or another URL, but we used DropBox)

Our Web App pulls the data from Dropbox URLs and caches it. Works pretty nice.

<appSettings>
<add key="url.playerUrl" value="https://dl.dropboxusercontent.com/s/fancypantsguid/VideoStreams.xml" />
<add key="url.scheduleUrl" value="https://dl.dropboxusercontent.com/s/fancypantsguid/Schedule.xml" />
<add key="url.speakerUrl" value="https://dl.dropboxusercontent.com/s/fancypantsguid/Speakers.xml" />
<add key="Microsoft.ServiceBus.ConnectionString" value="Endpoint=sb://[your namespace].servicebus.windows.net;SharedSecretIssuer=owner;SharedSecretValue=[your secret]" />
</appSettings>

The code is simple, as code should be. Wanna show the schedule? And yes , it's a TABLE. It's a table of the schedule. Nyah.

@foreach(var session in schedule) {
var confTime = session.Time;
var pstZone = TimeZoneInfo.FindSystemTimeZoneById("Pacific Standard Time");
var attendeeTime = TimeZoneInfo.ConvertTimeToUtc(confTime, pstZone);
<tr>
<td>
<p>@confTime.ToShortTimeString() (PDT)</p>
<p>@attendeeTime.ToShortTimeString() (GMT)</p>
</td>
<td>
<div class="speaker-info">
<h4>@session.Title</h4>
<br>
<span class="company-name"><a class="speaker-website" href="/speakers.cshtml?speaker=@session.Twitter">@session.Name</a></span>
<br>
<p>@session.Abstract</p>
</div>
</td>
</tr>
}

Scaling Out

Scaling DotNetConfWe've been on an extra small Azure Website and then switched to two large (and finally, two medium as large was totally overkill) web sites. 

We scale up (and hence, pay) only during the conference and turn it down to Small when we're done. No need to spend money if we don't need to.

Scaling DotNetConf to Large

Updating the Site in Real-time with SignalR

Because the YouTube link changes with each Hangout, we had the problem that attendees of the conference would have to hit refresh themselves to get the new URL. There's a number of solutions to this that I'm sure you're already thinking about. We could meta refresh, refresh on a timer, but these aren't on demand. We also wanted to show a few videos during the downtime. One of us preps the next speaker while the other queues up videos to watch.

We realized this was a problem at about 10pm PST last night. Javier and I got on Skype and came up with this late night hack.

What if everyone had SignalR running while their were watching the videos? Then we could push out the next YouTube video from an admin console.

So visualize this. There's the watcher (you), there's a admin (me) and there's the server (the Hub).

The watcher has this on their main page after including the /signalr/hub JavaScript:

$(function () {
var youtube = $.connection.youTubeHub;
$.connection.hub.logging = true;

youtube.client.updateYouTube = function (message, password) {
$("#youtube").attr("src", "http://www.youtube.com/embed/" + message + "?autoplay=1");
};
$.connection.hub.start();

$.connection.hub.disconnected(function () {
setTimeout(function () {
$.connection.hub.start();
}, 5000);
});
});

The watcher is listening, er, watching, for a SignalR message from the server with the YouTube video short code. When we get it, we swap out the iFrame. Simple and it works.

Here's the admin console where we put in the next YouTube code (I'm using Razor in ASP.NET Web Pages in WebMatrix, so this is mixed HTML/JS):

<div id="container">
<input type="text" id="videoId" name="videoId"><br/>
<input type="text" id="password" name="passsword" placeholder="password"><br/>
<button id="playerUpdate" name="playerUpdate">Update Player</button>
</div>

@section SignalR {
<script>
$(function () {
var youtube = $.connection.youTubeHub;
$.connection.hub.logging = true;

$.connection.hub.start().done(function () {
$('#playerUpdate').click(function () {
youtube.server.update($('#videoId').val(), $('#password').val());
});
});
$.connection.hub.disconnected(function() {
setTimeout(function() {
$.connection.hub.start();
}, 5000);
});
});
</script>
}

We put in the short code, the password and update. All this must be complex eh? What's the powerful SignalR backend up running in the cloud backed by the power of Azure and Service Bus look like? Surely that code must be too complex to show on a simple blog, eh? Strap in, friends.

public class YouTubeHub : Microsoft.AspNet.SignalR.Hub
{
public void update(string message, string password)
{
if (password.ToLowerInvariant() == "itisasecret")
{
Clients.All.updateYouTube(message);
ConfContext.SetPlayerUrl(message);
}
}
}

This is either a purist's nightmare or a pragmatists dream. Either way, we've been running it all day and it works. Between talks we pushed in pre-recorded talks and messages, then finally when the live talk started we pushed that one as well.

We also updated the DropBox links with the current Video Stream so that new visitors showing up would get the latest video, as new site visitors wouldn't have been connected when the video "push" message went out.

image

What about scale out? We sometimes have two machines in the farm so we need the SignalR "push updated youtube video message" to travel across a scale-out backplane. That took another 10 minutes.

Scaling out with SignalR using the Azure Service Bus

We used the SignalR 1.1 Beta plus Azure Service Bus Topics for scale out and added an Azure Service Bus to our account. Our app startup changed, adding this call to UseServiceBus():

string poo = "Endpoint=sb://dotnetconf-live-bus.servicebus.windows.net/;SharedSecretIssuer=owner;SharedSecretValue=g57totalsecrets=";   
GlobalHost.DependencyResolver.UseServiceBus(poo,"dotnetconf");
RouteTable.Routes.MapHubs();

Now SignalR uses the Service Bus Topics for "Pub/Sub" to pass notifications between the two web servers. I can push a new video from Web 1 and it is sent to everyone on Web 1 and Web 2 (or Web N) via SignalR's realtime persistent connection.

image

We'll delete this Service Bus Topic as soon as we are done. I would hate for the bill to get up into the nickels. ;) Here's example pricing from the Azure site:

432,000 Service Bus messages cost 432,000/10,000 * $0.01 = 44 * $0.01 = $0.44 per day.

I'm not sure how many messages we've done, but I can rest assured it won't cost a pile, so that's a relief.

Thank you to the Community!

  • Big thanks to designer Jin Yang who created the dotnetConf logo and design. Tweet @jzy and tell him you think he is special. Thanks to Dave Ward for converting Jzy's design into HTML!
  • Kudos and thanks to Javier Lozano for his coding, his organizing, his brainstorming and his tireless hard work. It was also cool for him to sit with me for hours last night while we hacked on SignalR and the DotNetConf.net site.
  • Thanks to David Fowler for saying "it'll just take 10 minutes to add Service Bus." 
  • Thanks to Eric Hexter and Jon Galloway for their organizational abilities and generous gifts of time on all the *conf events!
  • But mostly, thanks to the speakers who volunteered their time and presented and the community who showed up to watch, interact and play with us!

Sponsor: The Windows Azure Developer Challenge is on.  Complete 5 programming challenges for a chance at spot prizes, Stage prizes and the Grand Prize. Over $16,000 is up for grabs with 65 chances to win!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. I am a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Sponsored By
Hosting By
Dedicated Windows Server Hosting by ORCS Web
Friday, April 26, 2013 1:01:52 AM UTC
Brilliant, Scott...just brilliant. Your posts are SO full of great info.
Patrick Gaul
Friday, April 26, 2013 3:14:58 AM UTC
That was great, Scott. Thank you!
Friday, April 26, 2013 3:59:18 AM UTC
"We've been on an extra small Azure Website"

I thought the smallest reserved instance your could use for Websites was a small? It'd be great if extra small instances were available!
Friday, April 26, 2013 5:25:41 AM UTC
@JohnClayton - You're right, it was a small instance. Also, we ran it as Shared for a while since the traffic wasn't as heavy.
Friday, April 26, 2013 8:17:36 AM UTC
Great post. Really interesting. Enjoyed yesterday and looking forward to today. Appreciate all you guys do in and for the dev community.
Friday, April 26, 2013 11:57:01 AM UTC
It is all so simple and easy.

One thing, is there a small race condition there if somebody loads the page just as you are switching from one video to another? So they pick up the old xml just prior to it getting updated but don't receive the signalR message as they come in just after it was sent. Just being picky I know, it was something you had to put together very quickly and it's impressive.
Mark Adamson
Friday, April 26, 2013 12:01:33 PM UTC
great info and tips!
I've been using Hangouts for seom Q&A and tech talks, so easy and youtube is so big, makes everyone's life easier.
people can go to XBOX, to our channel, and watch something real time.

My only problem with hangouts going directly to the channel is the lack of analytics, you don't have ways of knowing how many people are watching this on demand. this would be very nice. besides that, it's brilliant.
rodolpho
Friday, April 26, 2013 12:16:27 PM UTC
This could lead to many small, niche conferences popping up all over the web. Ultimately this is great for developers and other professionals with specific interests. I'm inspired to tinker with Google Hangouts now!
Friday, April 26, 2013 1:09:41 PM UTC
Scott,
could you please take care of taping the speakings & conversations on a youtube channel *with* approved sound-quality/setup?
In the past, there were youtube snippets with really bad audio/sound-level/setup, with the result that you couldn't watch (~"hear") it with fun, since the sound was really broken.
Friday, April 26, 2013 2:56:23 PM UTC
MarkAdamson - Yes, you are right. We thought of that too and addressed with line 8 of the YouTubeHub class. We take the video ID and store it in cache. The Dropbox thing is a backup incase of AppPool restarts.
Friday, April 26, 2013 4:17:46 PM UTC
Javier. Thanks, sounds like you have it covered. What happens if they load the page between lines 7 and 8 executing? I guess I don't know exactly what the ConfContext does, but it seems like the signalR client must query it after start-up or something.
Mark Adamson
Friday, April 26, 2013 9:17:25 PM UTC
Are you scaling up and down manually? It looks like azure auto scaling is pretty easy to configure, although I haven't used yet. I would love to see an post about that.
Friday, April 26, 2013 10:55:17 PM UTC
This is amazing! Thanks so much! I'm inspired to get some friends and hold a web series conference this way!

Rob
Friday, April 26, 2013 11:09:41 PM UTC
Vicente - We are doing it manually because it's simple. Good to know there are autoscale options, though!
Saturday, April 27, 2013 11:47:26 AM UTC
Short of having polls and requiring attendees to register, you've got a great replacement for GoToWebinar with limitless connections that runs on every device that supports YouTube.
Saturday, April 27, 2013 10:38:02 PM UTC
Hi Scott,

Funnily enough Jeff's comment describes exactly what I was looking at, namely extending this so that anyone can run their own conference and stream/upload the video. I was thinking of doing it as part of the Azure developer challenge and put a brief article at http://www.codeproject.com/script/Articles/ArticleVersion.aspx?waid=71670&aid=584534.

Are you ok with me doing this, or are you already looking at doing this anyway. If so, I can withdraw my article, but I thought it was something worth looking into if you weren't already.

Thanks,
Phil
Phil Lee
Sunday, April 28, 2013 8:24:00 AM UTC
I only had a little time to watch but it did look really interesting. The only downside for me was that Google started streaming adverts and did not want to stop.

I hit refresh and I got back to the live feed which was disconcerting, I'd assumed that I was getting the adverts because you were switching streams but it seems Google just decided to show them for some other reason.

Not a criticism as such, I only mention it because I'm pretty sure you were unaware of it and would want to know.
Monday, April 29, 2013 2:38:03 AM UTC
This inspired me to whip up a "live blog" app for one of my sites to cover a, well, a live event next week. It's all working brilliantly fast, except the messages aren't being picked up by the other instances via service bus. I can see the messages are flowing through it, but the SingalR broadcast from the other instance isn't happening. To test this, I RDP'd into each instance, and brought up the instance by IP in IE. I triggered the message in one, and it delivered to the local machine, but not the other. Not sure what I'm missing on this one.
Monday, April 29, 2013 3:59:07 PM UTC
I just wanted to say that the use of Google Hangouts with SignalR is really inspired genius. Offload all of the video streaming and recording which still maintaining the ability to separate the sessions.
Wednesday, May 01, 2013 8:32:28 PM UTC
Did it, in fact, take 10 minutes to add Service Bus?
Friday, May 03, 2013 8:03:21 AM UTC
If you can put the code on github we would love to play with it and may be initiate conf like this for local groups in different countries.

Loved the dot net conf. downloaded all the sessions and missed you as speaker in the conf.
Sunday, June 23, 2013 1:14:33 AM UTC
Scott,

Why you needed to create new hangout for each presenter? You could just invite all presenters and mute/unmute them? Or just ask them to close hangout when they finish?
Monday, June 24, 2013 7:58:22 PM UTC
We created a new one because each new Hangout becomes a *separate* YouTube episode. We didn't want the whole conference to be one GIANT 8 hour show.
Monday, June 24, 2013 8:21:49 PM UTC
But at the end you could cut each show and save as 8 separated episodes. And then remove 8h video. Am I right?

I am just asking if you experienced (or heard about) some issues with recording 8h long hangout.
Monday, June 24, 2013 8:27:47 PM UTC
No, once a Live Hangout is up on YouTube, it can't be edited, it can only be trimmed.
Tuesday, June 25, 2013 1:16:37 AM UTC
Yeah, but if you 'trim' and 'save as' then you have original and new(trimmed) video. I have just tested it.
Tuesday, June 25, 2013 3:17:49 AM UTC
I stand corrected! Thanks for the info. I still think it's easier to do a separate one for each show.
Tuesday, July 09, 2013 3:12:20 PM UTC
The Sponsor link to the Azure Code Challenge just seems to bring me to this page?
Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.