I pushed an initial drop of code to hook up the Kinect to a WPF application using the Kinect SDK. The code is committed to BitBucket in the Kinect.WPF project. The main goal is to have a way to trigger UI events from body movements. The SDK doesn't install any kind of input device so there is no "official" way to translate the user movements into UI interaction, such as with a mouse device or a stylus.

The main structure of the project is to hook the KinectConnection to the window as an attached property. The KinectConnection takes in a Sensor, which is a wrapper around the Kinect device which tracks the skeleton frame. It also takes a number of KinectCursors, which are in effect a specific joint mapped as a cursor moving over the UI and triggering events. You can wire it up like so:

XAML

<Window x:Class="Kinect.App.MainWindow"
		xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
		xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
		xmlns:Tools="clr-namespace:Kinect.Tools;assembly=Kinect.Tools"
		Title="MainWindow"
		Height="600"
		Width="800">
	<Tools:KinectConnection.Connection>
		<Tools:KinectConnection>
			<Tools:KinectConnection.Sensor>
				<Tools:Sensor />
			</Tools:KinectConnection.Sensor>
			<Tools:KinectConnection.Cursors>
				<Tools:KinectCursor x:Name="rightCursor"
									Joint="HandRight" />
				<Tools:KinectCursor x:Name="leftCursor"
									Joint="HandLeft" />
			</Tools:KinectConnection.Cursors>
		</Tools:KinectConnection>
	</Tools:KinectConnection.Connection>
</Window>
	

As you can see it is hooked up as a normal attached property. The sensor is a simple instance, and the cursors are linked to a joint by setting the Joint property. Internally the KinectConnection attempts to determine the current window as work from there. The movements are defined relative to the window.

WPF makes it reasonably easy to create your own UI events by declaring RoutedEvents. Currently the RoutedEvents declared by the KinectCursor include CursorEnter and CursorLeave, as well as CursorUp and CursorDown. These are just the initial events, and I hope to be adding more in the future. More important are the combinations of cursor movements into gestures, such as when hands move together in a certain way, such as for example a pinch movement with the hands.

In the sample app in the Kinect.WPF project the UI is set to react to the KinectCursor entering and leaving. In the project this is done by setting an EventTrigger in the control style, but a handler could be attached in other standard ways as well. An example for the textblock is:

XAML

<Style x:Key="textStyle"
		TargetType="TextBlock">
	<Style.Triggers>
		<EventTrigger RoutedEvent="Tools:KinectCursor.CursorEnter">
			<BeginStoryboard>
				<Storyboard>
					<DoubleAnimation To="28"
										Duration="0:0:0.5"
										AccelerationRatio="0.10"
										DecelerationRatio="0.25"
										Storyboard.TargetProperty="(TextElement.FontSize)" />
				</Storyboard>
			</BeginStoryboard>
		</EventTrigger>
	</Style.Triggers>
</Style>	
	

As you can see, on the CursorEnter event the font size is changed.

This is very close to the current way that the mouse interacts currently. The main difference is obviously that there can be multiple KinectCursors. The cursors also need to expose more detailed motion data. This is what has to be utilized to define motion gestures, which is the bigger goal of the project. Follow the project for updates on this.

I'm just getting started on the project, and if you want to fork the code, feel free. I should also give due credit to the Kinect Paint project, which has inspired some of the code.