Difference between revisions of "Using Tracking Data from Depth Sensors"

Latest revision as of 05:42, 20 November 2015

Overview

Note: Depth sensors provide body tracking data based on native tracking algorithms. The accuracy of such tracking is not enough for professional-level motion capture solutions. Tracking provided by iPi Mocap Studio is based on iPi Soft's algorithms and do not use tracking data from sensors. However, sensor's tracking can be effectively used to improve workflow.

We record and use tracking data from Kinect 2 sensors. The data are used for:

• Detecting number of actors
• Detecting actor heights
• Detecting T-Poses and creation of takes
• Setting initial pose for detected T-poses
• Setting initial pose for the beginning frame of the Region-of-Interest
• Warning on presence of people in sensor's view during recording of the background

Recording Sensor Tracking Data

T-Pose in first sensor

iPi Recorder records tracking data provided by depth sensor for further use in iPi Mocap Studio.

Note: As soon as native tracking algorithms work well only when actor faces the sensor and each sensor works independently from others, for T-Pose and takes creation we only use data from the first sensor.

To make the most of sensors' tracking data, please follow this recommendations during recording:

• Strike T-Pose before each take
• You have to stay in T-Pose at least for one second
• Face the first sensor in T-Pose
• If there are several actors, each of them should face the first sensor

Tip: In case when capture area does not allow to face the sensor, you can stay at an angle to the sensor. Make sure that arms and legs are fully visible to the sensor.

Note: If you stand back to a sensor, native tracking will detect gaze direction opposite to the actual, so automatically detected initial pose will be incorrect and you'll have to manually fix this in iPi Mocap Studio.

Using Sensor Tracking Data

Detecting the Number of Actors and Actor Heights

When creating a new project from recorded .iPiVideo file, you need to specify the number of actors and their heights. Initial values for these fields are set based on tracking data. You just need to check them during project creation, as native tracking algorithms are not very accurate. Please correct the numbers if necessary.

Setting number of actors

Setting actor height

Note: If you only have a single depth sensor, you will not have a step for selection of the number of actors, as multiple actors tracking is not supported for a single depth sensor.

Note: Actor's gender is not detected by sensor, so please set it manually for female actors.

Creating Takes and Setting Actor Poses

Takes created automatically

• On project creation takes are created based on detected T-Poses. Each T-pose is considered as the beginning of a take.
• Region-of-Interest is set to the first take
• Actor's pose is set based on the first sensor's tracking data in the beginning frame of the Region-of-Interest
• You can manually adjust takes and the Region-of-Interest
• When you change the beginning frame of the Region-of-Interest, actor's pose will be set from sensor's tracking data in the beginning frame
• You must perform Refit Pose in the beginning frame of Region-of-Interest before you start tracking

Tip: Pose can be set from sensor's tracking data only if you have not performed tracking or any manual editing of the pose in this frame. Otherwise pose will not be changed.

Note: When you change the beginning frame of the Region-of-Interest, poses previously set from sensor's tracking data are not deleted. They will not affect tracking unless you perform Refit Pose in this frame. So there's no need to manually delete them.

Tip: To get better result, you need to adjust actor's proportions before you start tracking.

Note: If actor stands back to the first sensor, his gaze direction will be set opposite to the actual, and the entire pose will be incorrect. You can delete it and reset manually.

Tip: If pose is not accurate enough and Refit Pose does not fix it, you need to adjust pose manually to ensure it is close to actual depth data.