Dance Dance Revolution and the need for Alternate Human Computer Interaction
Z and I went over to my buddy John's yesterday to borrow his XBox360 copy of Dance Dance Revolution. I've been avoiding this game for nearly ten years. Partially because I can actually dance in real life (my wife can attest to this, and as she's an African, it's high praise, let me tell you) and DDR doesn't seem like dancing to me, but also because I know I'd get too into it.
But, John talked me into it and I talked my wife, begrudgingly, into playing it once. We didn't stop until 30 songs later. To quote my wife, "I won't stop until I get an 'A'!" She did, on Rapper's Delight, by the way.
I think that we took to this game for the same reason we took to the Wii - it's so physically interactive. After twenty years of twitch-gaming with a controller and my thumbs, and fifteen years of typing for a living, the last thing I want to do is come home and use a controller.
What so funny to me about Dance Dance Revolution to me is that it's just another version of Simon - a frenetic real-time psycho game of Simon set to house music while dancing with much gnashing of teeth - but Simon nonetheless.
Really immersive gaming (computing) requires one of two things. Either...
...a controller that mirrors something in reality...
Guitar Hero, another game I played at John's (loan that to me also, John!), is basically the same thing - it's Dance Dance Revolution with a fret. Addictive, surely, and I'm sure the wife will get a kick out of it, but it's a little too close to typing for these hands.
The original Xbox had an amazing (and VERY expensive) game called Steel Battalion that included a custom controller with over 40 buttons. It was brilliant. We played it at work (at lunch) for months. It was the most immersive gaming experience I'd had short of going to an arcade and getting into one of those $100K flight simulators. Of course, the controller was $200 - as much as an Xbox.
We've got to break out of this Mouse and Keyboard rut we're in as a culture (this includes you Quake and Unreal Tournament folks who insist a mouse and keyboard is the Only Way to Play - you're not helping!) and move into the Minority Report Multi-Touch Interaction world that folks like Jeff Han are pioneering.
...or intuiting intent via hand gestures
Now, $30k for something like Jeff Han's solution is (currently) untenable, but surely with all these Web Cams along with the brilliance of folks like Ashish and his Gesture Recognition stuff or the amazing uMouse stuff that Larry Lart (seriously, rush over there, now and read what he's doing) is working on (Hopefully I'll get a review of his stuff and he'll get a download link up soon. He said it'll be in beta the next week or so.) should give us some reliable moving of windows using our hands.
I figure it can't be that hard to watch a hand with a webcam, and that combined with the fact that there's only so many windows at a time on the screen - that lowers/narrows the number of things you'd want to do. Fine control of the mouse via gestures is a start, but a REALLY compelling solution would augment the mouse by using the web cam to track your hands and allow the push windows from monitor to monitor in a multimon scenario, minimize windows, launch Google, etc.
Certainly we could make it even easier by putting things on our hands (video of Atlas Gloves and Google Earth) and making the functionality very specialized.
Frankly I'm surprised that BillG would put so much work into Voice Recognition and Tablet but miss out on the opportunity to revolutionize User eXperience via a simple webcam. I don't need my webcam or computer to tell me if I'm sad - I'd like it to recognize my intent and act on it.
There's lots of Alternative Pointing Devices to choose from, but they all fall into the same tired metaphors. As mice go, personally I really like the Evoluent Vertical Mouse (it doesn't turn your hand unnaturally) and I also have Tablets for all my computers. I love my Ergodex, but I don't use it for coding like I used to. I'm convinced that gestures are the next big thing - and no, I don't mean mouse gestures. It's a huge shame that the Fingerworks guys went out of business. Their pad was brilliant. I know Rory is a huge fan of their iGesture pad. It was just too expensive. All these mechanisms are tired shadows of what a good gesture system could do.
Making cheap webcams recognize our hand gestures - even if I have to point the camera at my hand on the mouse - has huge potential. This is a 100% solvable software problem.
Of course, the real tragedy in all this? I'll never dance (or, "Dance Dance" rather) as well as this five-year-old.