n0n4m3's homepage

Part of my Kinect Portfolio presentation had slides controlled by gestures. This short tutorial explains how I made it work. As I said in the other post, I used a Gesture Library to help with the gesture recognition, Kinect SDK Dynamic Time Warping (DTW) Gesture Recognition. This is a great project that is available on CodePlex and uses a cool algorithm called Dynamic Time Warping which is pretty cool for pattern recognition such as gestures or voice, among others. If you’re curious like me and want to know how this algorithm works, take a look at this link.

But onto the slide tutorial. I started by taking the core files to handle the gesture recognition and plug it in with my project. After a couple of little changes mostly to call an event handler when a gesture is recognized and to work with the final Kinect SDK 1.0., I used the original program to generate the gestures file which contains the information needed to detect gestures. I actually recorded two different gestures for each “SlideLeft” and “SlideRight” gesture just to make sure that different gestures done by different people would be caught by the algorithm.

The next step was to figure out how to animate the slides. I didn’t have experience with WPF animations but I arrived at this solution which works but might not be the best.

VisualStates and their transitions

I created 3 visual states to handle the slide left and slide right: previous (VisualState 1), current (VisualState 2) and next (VisualState 3). There are 2 visual transitions (left and right) between each 2 states and every time we change from one visual state to another we update the “next” target with new slides.

To automatize the transitions I decided to create a SlideManager that starts by reading all the “slides” (Grid elements) under a parent element and initializing the visual states by adding the event handlers. I created a simple structure to keep track of slides and transitions between slides because initially I wanted to include vertical transitions by ended up not doing them.

// Initialize slides.
for (int i = 0; i < mainSlideContainer.Children.Count; i++)
{
	Grid slide = mainSlideContainer.Children[i] as Grid;
	if(slide == null)
		continue;

	Slide s = new Slide()
				            {
				                Name = slide.Name,
				                GUIElement = slide,
				            };

	m_slides.Add(s);
}

for (int i = 0; i < m_slides.Count; i++)
{
	Slide nSlide = null;
	Slide pSlide = null;

	if (i < m_slides.Count - 1)
		nSlide = m_slides[i + 1];
 	if (i > 0)
		pSlide = m_slides[i];

	Slide s = m_slides[i];
	s.Transitions = new Dictionary();
	if(nSlide != null)
		s.Transitions.Add(Transition.RightToLeft, nSlide);
	if (pSlide != null)
		s.Transitions.Add(Transition.LeftToRight, pSlide);
}

There are two other important methods that change slides: MoveToNextSlide() and MoveToPrevSlide(). Each method checks if the current slide has the transition we want to perform, updates the current slide and visual state attribute members and calls the method that actually performs the transition.

public void MoveToNextSlide()
{
	if(m_inTransition)
		return;

	Slide nextSlide;
	if (m_curSlide.Transitions.TryGetValue(Transition.RightToLeft, out nextSlide))
	{
		m_prevSlideIndex = m_curSlideIndex;

		m_curSlideIndex = Math.Min(m_curSlideIndex + 1, m_slides.Count - 1);
		m_curSlide = m_slides[m_curSlideIndex];
		m_curVSIndex = (m_curVSIndex + 1) % m_visualStates.Count;
		m_curVS = m_visualStates[m_curVSIndex];

		m_inTransition = true;
		ExtendedVisualStateManager.GoToElementState(m_vsManagerContainer, m_curVS.Name, true);
	}
}

Another part of the code that is worth mentioning is the two event handlers at the end of this manager: Storyboard_Completed and VSGCurrentStateChanged.

private void Storyboard_Completed(object sender, EventArgs e)
{
	m_slides[m_curSlideIndex].GUIElement.Opacity = 1;
	m_slides[m_curSlideIndex].GUIElement.Visibility = Visibility.Visible;

	if (m_prevSlideIndex != m_curSlideIndex)
	{
		m_slides[m_prevSlideIndex].GUIElement.Opacity = 0;
		m_slides[m_prevSlideIndex].GUIElement.Visibility = Visibility.Collapsed;
	}
}

The visual state transitions do the slide movement and the storyboard controls the opacity and visibility of the element in that state. Whenever we change from one state to the other we have to reset the opacity and visibility properties so that the next time we do the transitions the new slides have the correct properties.

private void VSGCurrentStateChanged(object sender, VisualStateChangedEventArgs e)
{
	VisualState nvs = m_curVS;

	// update the next visual state time line (only if necessary)
	if (m_curSlideIndex < m_slides.Count - 1)
	{
		Timeline tl = FindObjectByName(nvs.Storyboard.Children, nvs.Name + "_next" + TLOpacName);
		if (tl != null)
			tl.SetValue(Storyboard.TargetNameProperty, m_slides[m_curSlideIndex + 1].Name);

		tl = FindObjectByName(nvs.Storyboard.Children, nvs.Name + "_next" + TLVisName);
		if (tl != null)
			tl.SetValue(Storyboard.TargetNameProperty, m_slides[m_curSlideIndex + 1].Name);
	}

	m_inTransition = false;
}

The same goes for the storyboard target name property, whenever we change to a new state we have to update the next slide. I also used a flag to prevent it from starting a new transition while we’re still doing the animation from the current transition.

The KinectManager is a simple class to handle the Kinect’s initialization that I adapted from the samples. It’s far from complete as it is but it serves well for this tutorial.

You may ask yourself why go through all this trouble when there is PowerPoint and Kinect applications to do the slide animation controls? Firstly, because it is fun to explore and learn new things and secondly because there are parts of the complete interactive portfolio that weren’t so easy to do in PowerPoint such as the particle system or the pong game.

Note: The transitions between texts were programmed in a similar way but instead of using 3 Visual States I only used 2, one for display the text and the other and to hide it, then it’s just a matter of setting the right text when you do the transitions.

Download
Kinect Slide Tutorial (requires: .Net4, Kinect SDK 1.0, VS2010 and up)