Multiple Actors

Note: Feature is available in Pro edition only.

You can try this functionality in other editions but in this case all results will be saved in restricted trial format which prohibits export of animation.

Introduction

The software supports multi-person tracking with the following limitations:

• Currently two or three actors tracking is available in the public release.
• To track multiple actors, at least 3 Sony PS Eye cameras per actor are required
  • It's recommended to use 4 Sony PS Eye cameras per actor. Having more cameras minimizes actor's mutual occlusion and improves tracking reliability.
  • For two actors you can use 3 depth sensors
  • For simple dual actor actions when actors are rarely close to each other you can use 4 — 5 RGB cameras or even two depth sensors.

Actors Clothing

For color cameras:

• All regular recommendations.
• Plus it's highly recommended to use different colors for clothing of different actors. It will minimize tracking errors.

For depth sensors:

• Only regular recommendations. No additional recommendations, because current version of iPi Mocap Studio doesn't use color data from depth sensors during tracking (color data is used only for visualization).

Cameras Configuration

The main requirement is to minimize self- and mutual occlusions of actors. In general, cameras should be placed in full-circle with at least two cameras at 2.5+ meters height. If you're going to use minimum recommended number of cameras (or less) then all actions should be planned in such a way which minimize mutual actors occlusion. For example, screenshot below demonstrates how to capture two actors handshake with just two depth sensors:

Scene Configuration Example

T-pose

It's convenient but not strongly required all actors to stay in T-pose at one and the same time as far from each other as possible (actors should be fully visible in maximum number of cameras).

T-poses of Actors

Processing

• During project creation you will need to specify number of actors and then parameters of each actor:

How many actors?

First actor parameters

Second actor parameters

• At the toolbar you will see buttons for switching between actors. Also you can switch by clicking on actor model in the 3D view port. Skin of "inactive" actor is grayed. Content of Actor tab is displayed for the active actor.

Active actor on UI

• For each actor:
  1. Roughly align actor's model with actor's images in all cameras using Move, Rotation and IK tools
  2. Adjust actor parameters if needed
  3. Click Analyze Actor Appearance button
  4. Correct actor's colors with the aid of Eye dropper if needed
  5. Click Refit Pose on Tracking tab.

     Note. Unlike all other content of Tracking tab this button runs refit pose for active actor only and doesn't affect the inactive one

  6. Repeat steps starting from 3 if needed

• Adjust Region-of-Interest, choose tracking settings on the Tracking tab and execute tracking

  Note. All options and action buttons on the Tracking tab except Refit Pose button are applied to all actors.

Tracking Tab Controls

• After processing you can use regular procedures for Manual Clean-up and Automatic Refinement and Filtering.

Motion Controllers

You can use up to 3 motion controllers per actor for:

• left hand
• right hand
• head

For projects with two actors you will be able to choose both body part and actor index in the Attach to drop-down list on Motion Controllers tab:

File:Two actors motion controllers.png Attach to drop-down list

Animation Export

Using Import target character on Export tab you can import target character for active actor. If you want to import target character for the second actor, just make it active using buttons on toolbar and repeat import. Target characters can be the same or different.

In a similar way, Export animation button works for active actor. That is you can export animation for each actor separately. Current version of iPi Mocap Studio doesn't allow to export animation for both characters to the one output file.

Export

Sample Video and Project

Here you can find sample videos and projects for three actors captured with ten Sony PS3 Eye cameras.

                  

Calibration Sequence

Sample calibration video file and appropriate iPiCalib project which is result of processing in iPi Mocap Studio:

			↓ Download video file ↓ three_actors_10_cams_catch_and_throw_calib.iPiVideo 436 MByte
			↓ Download calibration file ↓ three_actors_10_cams_catch_and_throw_calib.iPiCalib 363 KByte

How to use:

• Download and install iPi Mocap Studio.
• Download both above files and put them all in one and the same folder on your hard drive.
• Run iPi Mocap Studio, click Open button or press Ctrl+O and select three_actors_10_cams_catch_and_throw_calib.iPiCalib file downloaded on the previous step.
• Now you can "play" with resulting calibration data in iPi Mocap Studio.
• If you want to try to process source video file by yourself then:
  • click New button or press Ctrl+N;
  • select three_actors_10_cams_catch_and_throw_calib.iPiVideo file downloaded before;
  • choose Calibration project type;
  • switch to Calibration tab and push Calibrate button;
  • to adjust scene scale use the following parameters:
    • the height of first camera over ground plane is about 1.41 m

Performance Sequence

Sample performance video file and appropriate iPiMotion project which is result of processing in iPi Mocap Studio:

			↓ Download video file ↓ three_actors_10_cams_catch_and_throw_action.iPiVideo 333 MByte
			↓ Download project file ↓ three_actors_10_cams_catch_and_throw_action.iPiMotion 2.63 MByte

How to use:

• Download and install iPi Mocap Studio.
• Download both above files and put them all in one and the same folder on your hard drive.
• Run iPi Mocap Studio, click Open button or press Ctrl+O and select three_actors_10_cams_catch_and_throw_action.iPiMotion file downloaded on the previous step.
• Now you can "play" with resulting animation in iPi Mocap Studio, import your target character and export resulting animation based on the following instructions.