Introducing the Future - The Birth of the Matrix

Posted on Mon 29 Dec 2008 by Keira Peney under Industry News .
[Link]

Input methods. Always an iconic part of video gaming, the various joysticks, controllers, mice, keyboards, wiimotes, dance mats, stylus, and touch-screens have, over the years, covered everything from violent button-mashing to frustratingly intricate flight simulators. Custom input devices are now de-rigeour, with wii-fit and rock-band topping the charts in terms of expensive periphirals.

An input device is more than just a novelty light-sabre. It is our only means of connecting with the game-world. No matter how quick or skilled we are, that movement has to be interpreted through the device. Luckily for us, input devices have evolved quite considerably since the early days of gaming. However, analogue sticks simply cannot recreate the feel of a gun - or a sword - in your hands. No keyboard can come close to recreating the vast range of movement available to us.

Clunky controls are the quickest thing to frustrate us. If I can’t handle a game intuitively, I quickly give up on it. I’m sure this is true of most people - if we die, and it’s our fault, fine. If we die, and it’s because of some incredibly obtuse camera angle or a sudden reversal of polarities, we’re done. Yet many of what we find ‘easy’ is really just long-term habit. If all games use the right-stick for the camera, and the left-stick for movement, it’s very difficult to adjust to a game that switches them over.

Long term gamers take dual-analogue sticks and L1 and R2 buttons in their stride. It’s drilled into our brains, and we’ll pick up new games swiftly. For newcomers, it’s not so easy. Like a person first learning to type, they’ll hover, hesistantly, mumbling triangle-to-jump, triangle-to-jump under their breath. Watching someone new to gaming attempt to move their two thumbs indepedently of each other is a lot like watching someone trying to learn the guitar - first they will focus on one hand, and then on the other. Move - then look. Move - then look. In any online FPS match, this would result in near-instant death.

Any input method is a trade-off between complexity and intuitiveness. The more buttons and sticks you have to deal with, the more variety and choice you have, but the more you have to learn. The best input devices are the ones that closely mimic skills we already have - the DS stylus works because most of us have used pens and pencils. We know what to expect. The wii-mote works because it allows to mimic the movements we already expect - we wave it around and stab it if we’re sword fighting, we sweep it underarm if we’re bowling. It draws on muscle-memory we’ve already developed.

Yet even the wii-mote has to be learned. It’s easy, yes, but still requires a certain amount of wii-mote training.

What would happen if a game could mimic your every movement - accurately? Suddenly, the input method would be your own body - a device that you spend every moment of every day with, performing a huge variety of complex actions. From incredibly subtle twitches, to massively powerful gestures, most of us have access to a large range of movement.

It’s not a new idea - but until recently, the idea mainly consisted of lots of attached wires and expensive nodes. But now someone has come up with a system that only requires a video camera to work. It’s called XTR - and it’s functional.

It’s applicable outside of video games, of course. But for the gaming industry, it’s a very interesting development, because it brings virtual reality one step closer. Imagine World of Warcraft - but without the keyboard and mouse. Imagine you’re the one gesturing the magic spells, and hacking at the kobolds. Imagine Sim City, where you’re literally smoothing out the mountains with your hands.

Naturally, I can think of some drawbacks. So I was very pleased when one of the developers of XTR of XTR agreed to answer some questions about it. Purely from a speculative standpoint, we can see the quality of immersive storytelling that can emerge from such a technology. How much longer before we are actually inside the game?

To find out more about what I learned from Dor Givon about XTR, and some of his speculations as to where this can take us, tune in tomorrow.

7 Responses to “Introducing the Future - The Birth of the Matrix

wonderwhy-er Says:
December 30th, 2008 at 9:10 am

Hmm… I was wondering on such technologies but how they are doing it with only one camera? I tough that for good 3D input at least two are needed…

There is another technology I think has better quality tough it needs specific cameras… http://gizmodo.com/gadgets/zcam-depth-camera-could-be-wii-challenger/zcam-3d-camera-is-like-wii-without-wiimote-and-minority-report-without-gloves-334426.php

Seen such idea implemented few times. Such cameras use some kind of infrared or laser flash and then catches it with additional sensors on camera matrix. As a result such cameras input is not only RGB for each pixel but RGBD where D is depth value. So such camera catches 3D landscape of what it sees. I find it better then just 2D motion that one usual camera can capture…

Game developer… Says:
January 3rd, 2009 at 11:32 am

Hi wonderwhy-er

Creating 3D using two views is done with triangulation methods, some companies are using that, very similar to stereo cameras.
If XTR is successfully doing 3D motion capture using a single camera that is a major breakthrough (look at their website).
From a game developer point of view, the Z cam you linked to is using projected IR to create depth maps, the distance of every pixel from the camera, but it provides no information other than that. Unfortunately it is useless information with respect to human machine interface because the application doesn’t know what the depth input relates to – what the user is doing, is it a chair, hand, or even two people? So you need an extra layer of RGB analysis to know that, which brings us to point zero (3D or not).
I think that once you have 3D motion capture, you have depth information plus knowing what the user is doing in the image. In this way the application can really see the image.
Regarding better quality, the Z cam is really the most accurate solution available today.
But what can you do with micron resolution in human machine interface? It’s below people’s movement resolution. So it’s not relevant if the resolution is microns or millimeters. So… When you come to the point you need to trigger your application with something the user did, only relying on depth data is not the solution for you.
Game developer…

Aaron Says:
January 15th, 2009 at 4:21 pm

In Black & White, players use gestures to cast spells. I love it, but it can be frustrating when the software doesn’t recognize movements. Those gestures are accomplished via a mouse, but even with free motion there could be a steep learning curve and problems with the performance of commands.

The Wii has demonstrated that motion capture games can be tiring. Even healthy gamers often prefer relaxed gaming to something that makes them break a sweat. Still, the Wii continues to sell like hotcakes, doesn’t it?

Looking forward to your interview!

Write the Game » Introducing the Future II - Interview with XTR Says:
January 20th, 2009 at 1:05 pm

[...] month, I introduced you to XTR, the new single-camera input method. Today, I’m pleased to discuss the results of the [...]

Write the Game » Project Natal: The Next Step Says:
July 27th, 2009 at 3:34 am

[...] wasn’t so long ago I talked about XTR, the single-camera input method. Not that much later, Xbox announced Project Natal, which takes the [...]

Generic xanax. Says:
September 18th, 2009 at 7:23 am

Xanax….

Buy xanax. Order xanax online. Xanax. Overnight xanax. Online prescription xanax. Xanax addiction….

armpit sweat Says:
November 14th, 2009 at 3:51 pm

Sweating for no reason at all sucks.