»

Greymail game for Kinect

Greymail Kinect game, arms flailing lol

Microsoft were announcing new features for Hotmail for what they called Greymail. That’s all the unwanted newsletters and the like that’s not spam and filling up your inbox. So VCCP decided to make a Kinect game for the various Hotmail events that Microsoft hold at Universities and the like. Our Creative Tech had done a quick prototype and it was decided that the game would be written in ActionScript and published as an Adobe AIR app.

AIRKinect’s SelectableHandle runs a timer before confirming a hit

AIRKinect’s SelectableHandle runs a timer before confirming a hit

A quick massage while the game counts in

A quick massage while the game counts in

I was on lead coding duties, with the database integration handed over by a fellow dev. Games screens designed by Simon Bostock, with some assets like badges and sounds supplied by a third party.

A fancy widget for inputting initials by waving your hands about

A fancy widget for inputting initials by waving your hands about

The XML lists score ranges for achievement badges

The XML lists score ranges for achievement badges

 

Set up

It was time to boot into Boot Camp on my mac for some Windows 7 love. After installing the Kinect for Windows SDK I then installed Adobe Flash Builder. I’m usually an FDT coder, but version 5.6 wasn’t out yet and AIRKinect uses a .ane native extension to receive data from the Kinect.

 

Code base

I used a simple MVC pattern. It’s really similar to the Chedds game I built last year. You can get an idea about how it’s set up and all linked together below.

This is all wrapped up in an AIR app

This is all wrapped up in an AIR app

Essentially there’s an invisible button covering the whole screen. When the player reaches forward (measuring the hand to shoulder z distance) the coordinates are sent via a command and picked up by the game area. Then there’s a quick loop through list of onscreen targets with a bounding box only collision test. If the hand hits a target the state changes for that hand.

 

AirKinect & the hands

Using AirKinect to control the hands is relatively straightforward. Each hand has properties for:

  1. Active. The hand is extended forward.
  2. Locked. The hand has grabbed a target.

 

Information is read from the skeleton every frame (using AIRKinect’s SkeletonFrameEvent). The pad modifier is applied to let players reach the corners of the screen. The results are passed to CursorController to be picked up by various parts of the game. Here’s the onSkeletonFrame so you can see it’s mostly about reading the numbers and applying them to the game.

private function onSkeletonFrame(event:SkeletonFrameEvent):void
{
  var numSkeletons:uint = event.skeletonFrame.numSkeletons;
  …
  if (_skeleton != null)
  {
    _leftHandJoint = _skeleton.getJoint(AIRKinectSkeleton.HAND_LEFT);
    _rightHandJoint = _skeleton.getJoint(AIRKinectSkeleton.HAND_RIGHT);

    _rightShoulderJoint = _skeleton.getJoint(AIRKinectSkeleton.SHOULDER_RIGHT);
    _leftShoulderJoint = _skeleton.getJoint(AIRKinectSkeleton.SHOULDER_LEFT);

    var pad:Number = _cursorModel.pad;

    _leftHandJoint.x -= pad;
    _leftHandJoint.x /= (1 - pad) - pad;
    _leftHandJoint.y -= pad;
    _leftHandJoint.y /= (1 - pad) - pad;

    _rightHandJoint.x -= pad;
    _rightHandJoint.x /= (1 - pad) - pad;
    _rightHandJoint.y -= pad;
    _rightHandJoint.y /= (1 - pad) - pad;

    var m:Number = _cursorModel.distanceMultiplier;

    _leftHandCursor.update(_leftHandJoint.x, _leftHandJoint.y, _leftHandJoint.z);
    _rightHandCursor.update(_rightHandJoint.x, _rightHandJoint.y, _rightHandJoint.z);

    _xyLeft.x = _leftHandCursor.x * _sizeModel.width;
    _xyLeft.y = _leftHandCursor.y * _sizeModel.height;
    _xyRight.x = _rightHandCursor.x * _sizeModel.width;
    _xyRight.y = _rightHandCursor.y * _sizeModel.height;
    _cursorController.leftHandPosUpdated(_xyLeft);
    _cursorController.rightHandPosUpdated(_xyRight);

    // test grabbing state
    var grabRight:Boolean = _rightShoulderJoint.z - _rightHandJoint.z > 0.2;
    var grabLeft:Boolean = _leftShoulderJoint.z - _leftHandJoint.z > 0.2;
	
    // apply grabbing results
    if ((grabRight || grabLeft) && !_cursorModel.rightHandActive && !_cursorModel.leftHandActive) _cursorController.startGrabbing();
    else if (!grabRight && !grabLeft && (_cursorModel.rightHandActive || _cursorModel.leftHandActive)) _cursorController.stopGrabbing();

    if (_rightHandCursor.enabled)
    {
      if (grabRight && !_cursorModel.rightHandActive) _cursorController.announceRightHandActive();
      else if (!grabRight && _cursorModel.rightHandActive) _cursorController.announceRightHandInactive();
    }

    if (_leftHandCursor.enabled)
    {
      if (grabLeft && !_cursorModel.leftHandActive) _cursorController.announceLeftHandActive();
      else if (!grabLeft && _cursorModel.leftHandActive) _cursorController.announceLeftHandInactive();
    }
  }
  …
}
Top ten are saved to a database in local storage

Top ten are saved to a database in local storage

 

It was a lot of fun using AIRKinect and even more fun testing the game. Hopefully next time I can build something for the Kinect with WPF :-).

Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>