Difference between revisions of "Multiple PS Eye Cameras Calibration"
(Created page with "{{:Calibration Definition}} === Importance of high frame rate === You should record calibration video at the same resolution as your action video and at the same (or higher)...") |
|||
| (33 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
| + | === Flashlight Calibration === | ||
| + | |||
{{:Calibration Definition}} | {{:Calibration Definition}} | ||
| − | === Importance of high frame rate === | + | ==== Importance of high frame rate ==== |
You should record calibration video at the same resolution as your action video and at the same (or higher) frame rate. | You should record calibration video at the same resolution as your action video and at the same (or higher) frame rate. | ||
| Line 9: | Line 11: | ||
Calibration at low frame rate may lead to reduced accuracy because of increased synchronization errors. | Calibration at low frame rate may lead to reduced accuracy because of increased synchronization errors. | ||
| − | === Glowing Marker === | + | ==== Glowing Marker ==== |
{{:Glowing Marker}} | {{:Glowing Marker}} | ||
| − | === Step 1: Running [[iPi Recorder]] in Calibration Mode === | + | ==== Step 1: Running [[iPi Recorder]] in Calibration Mode ==== |
Run [[iPi Recorder]] and choose one of the darkening modes in "darkening for calibration" list (for Sony PS Eye cameras) | Run [[iPi Recorder]] and choose one of the darkening modes in "darkening for calibration" list (for Sony PS Eye cameras) | ||
| − | + | [[File:Calibration-darkening.png|center|link=]] | |
| − | [[File: | + | |
or set Exposure to reasonably small value (for DirectShow-compatible web cameras) | or set Exposure to reasonably small value (for DirectShow-compatible web cameras) | ||
| − | [[File: | + | [[File:Calibration-exposure.png|center|link=]] |
This is important because it helps to reduce motion blur during calibration. | This is important because it helps to reduce motion blur during calibration. | ||
| Line 26: | Line 27: | ||
Video will look dim in calibration mode. | Video will look dim in calibration mode. | ||
| − | + | {{Important|Do not turn off the light in the room during calibration! This will not help the software but will make it harder for you to see what is happening on recorded video when you view it later.}} | |
| − | === Step 2: Record Calibration Video === | + | ==== Step 2: Record Calibration Video ==== |
Start video recording. | Start video recording. | ||
| Line 34: | Line 35: | ||
Move the marker slowly through your entire capture volume (front-top-right-bottom-left-back-top-right-bottom-left). Start from top and move the marker in a descending spiral motion. | Move the marker slowly through your entire capture volume (front-top-right-bottom-left-back-top-right-bottom-left). Start from top and move the marker in a descending spiral motion. | ||
| − | + | {{Tip|The exact trajectory of the marker is not so important, just try to cover the whole capture volume, or at least its perimeter.}} | |
| − | + | {{Tip|For successful calibration, several hundred frames are sufficient. Much more frames (like several thousands) will not produce more accurate results, but instead will increase the processing time and possible number of marker detection errors. So generally it is not recommended to record calibration videos of more than 1 minute length.}} | |
| − | + | {{Tip|You should make the marker visible to as much cameras as possible at all times. Hold the marker in the straight arm away from your body. In a circle configuration, when approaching the boundary of the capture area, position the marker inside the area, and your body outside.}} | |
[[File:Calibration1.jpg|center|link=]] | [[File:Calibration1.jpg|center|link=]] | ||
| Line 47: | Line 48: | ||
Put the marker to the ground at each corner and at the center of capture volume. At least 4-5 ground points are needed for the correct detection of the groundplane. | Put the marker to the ground at each corner and at the center of capture volume. At least 4-5 ground points are needed for the correct detection of the groundplane. | ||
| + | <center> | ||
| + | {| | ||
| + | |[[File:Calibration_ground1.jpg|center|400px]] | ||
| + | |[[File:Calibration_ground2.jpg|center|400px]] | ||
| + | |} | ||
| + | </center> | ||
| − | + | ==== Step 3: Stop Recording and Check Recorded Video ==== | |
| − | + | ||
| − | + | ||
| − | + | Check the video and make sure that: | |
* There is no significant motion blur (image of marker looks like a round spot rather than an ellipse or a luminescent line) | * There is no significant motion blur (image of marker looks like a round spot rather than an ellipse or a luminescent line) | ||
| − | * Most of the time marker is visible in | + | * Most of the time (80%-90% of all recording time) marker is visible in all cameras |
| − | === Step 4: Take Note of Height of Your First Camera Over the Ground. === | + | ==== Step 4: Take Note of Height of Your First Camera Over the Ground. ==== |
| − | Take note of height of your first camera over the ground. You will need this parameter later | + | Take note of height of your first camera over the ground. You will need this parameter later. |
| − | === Step 5: Process Calibration Video in [[iPi Mocap Studio]] === | + | ==== Step 5: Process Calibration Video in [[iPi Mocap Studio]] ==== |
| − | + | {{Tip|You can postpone processing calibration video until after you finished recording other videos (e.g. your action videos). However, it is a good idea to process your calibration video as soon as it was recorded because it helps you ensure that you have good calibration.}} | |
To process calibration video please do the following: | To process calibration video please do the following: | ||
| − | * | + | * Create new calibration project in [[iPi Mocap Studio]]: |
| − | * ''' | + | ** Press ''New'' button or select '''File > New Project''' menu item or use <tt>Ctrl+N</tt> (2) |
| − | * Set the | + | *:[[Image:Calibration_Start_Flashlight_1.png|center|800px]] |
| − | + | ** Choose '''Calibration project''' type in '''New Project Wizard'''. | |
| − | * | + | ** Set the diagonal '''Field of View''' (FOV) for your cameras. If this screen is not shown, this means FOV is already known and you do not need to manually specify it. |
| − | ** | + | *:{{Note|If you use Sony PS Eye or Logitech QuickCam 9000 cameras, leave the FOV value at the default 75 degrees.}} |
| − | * | + | <center> |
| − | ** | + | {| |
| − | ** Click | + | |[[Image:Select_Project_Type_Calibration.png|center|400px]] |
| + | |[[Image:Calibration_Select_Camera_FOV.png|center|400px]] | ||
| + | |} | ||
| + | </center> | ||
| + | * Adjust the [[Region-of-Interest]] to cover the part of video that contains the glowing marker (3). | ||
| + | * Change the size of light marker spot on video if needed. | ||
| + | ** '''Medium''' usually work well, but you may need '''Large''' if you use Sony PS Move for calibration, or '''Small''' in some cases. | ||
| + | * Normally, you need to use default calibration settings | ||
| + | ** Auto-detect initial camera positions turned on | ||
| + | *** If calibration fails to detect initial camera positions correctly, you can turn this checkbox off and set camera positions manually. But in most cases it will be easier to re-record calibration sequence. | ||
| + | ** Auto-adjust camera FOV turned off | ||
| + | *** Use this mode only if you did not know camera field of view requested during project creation (if it was requested) or if it was set by default (this may be recommended for particular camera models). | ||
| + | * Click '''Calibrate Based on Flashlight Marker''' button and wait while the system finishes calibration (4). | ||
| + | |||
| + | ==== Step 6: Check Calibration Quality ==== | ||
Resulting scene should look like this: | Resulting scene should look like this: | ||
| − | [[File:Calibration_result1. | + | [[File:Calibration_result1.png|center|800px]] |
| − | + | Make sure you have <span style="color:white; background:#ffd700; padding:5px;">Good</span> or <span style="color:white; background:#8cbf26; padding:5px;">Perfect</span> calibration result. | |
| − | + | {{Important|1=<span style="color:white; background:#e93f3f; padding:5px;">Failed</span> calibration is not recommended to use, as you will not be able to get accurate tracking results. However, sometimes '''Failed''' status can be misdetected. If detected marker positions are close to marker image on video in all frames for all sensors, you can use this calibration for tracking.}} | |
| − | + | ||
| − | + | ==== Step 7: Check Ground Plane ==== | |
| − | + | [[Image:Floor-Detection-1.png|thumb|334px|Manual Floor Detection]] | |
| − | + | Ground plane should be detected automatically. Ground points are marked by <span style="color:white; background:#ffd700; padding:5px;">yellow</span> color. | |
| − | + | * Make sure that ground points are actually near the ground plane. | |
| − | + | * If ground plane is detected incorrectly, select ground points manually. | |
| − | === Step | + | ** Expand '''Manual Calibration Adjustment''' expander. |
| − | + | ** First, unmark all ground points by pressing '''Clear all points''' button. | |
| − | Ground plane should be detected automatically. Ground points are marked by <span style="color: white; background | + | ** For each ground point, click on it in 3D view and press '''Mark as ground''' button. |
| − | + | ** You can cancel marking point as ground point by pressing '''Unmark ground''' button. | |
| − | + | ||
| − | If ground plane is detected incorrectly, select ground points manually. For each point, click on it in 3D view and press | + | |
| − | + | ||
| − | + | ||
{{Important|If you do not mark ground points then the ground plane is incorrect and there is no sense in using Foot tracking option and camera heights values.}} | {{Important|If you do not mark ground points then the ground plane is incorrect and there is no sense in using Foot tracking option and camera heights values.}} | ||
| − | === Step | + | ==== Step 8: Set Scene Scale Using Camera Height as Reference ==== |
Now cameras in your scene are properly oriented relative to other cameras and relative to ground plane. But you still need to find one more parameter: scene scale. | Now cameras in your scene are properly oriented relative to other cameras and relative to ground plane. But you still need to find one more parameter: scene scale. | ||
| − | Use | + | Use '''Сamera #1''' height over ground to set correct scene scale. |
| − | [[File:Calibration_scale1.png|center| | + | [[File:Calibration_scale1.png|center|800px]] |
| − | |||
| − | |||
* Save results to calibration project file or using '''Save scene...''' button on '''Scene''' tab (6). | * Save results to calibration project file or using '''Save scene...''' button on '''Scene''' tab (6). | ||
| + | [[Image:Depth_Project_Scene_Tab.png|center|link=]] | ||
| + | |||
| + | === Calibration Based on AI Joints === | ||
| + | |||
| + | {{:Calibration Definition AI}} | ||
| + | |||
| + | ==== Importance of high frame rate ==== | ||
| + | |||
| + | You should record calibration video at the same resolution as your action video and at the same (or higher) frame rate. | ||
| + | |||
| + | Calibration at a different resolution may lead to reduced accuracy because cameras usually have different minor distortions at different resolutions (caused by internal scaling algorithm). | ||
| + | |||
| + | Calibration at low frame rate may lead to reduced accuracy because of increased synchronization errors. | ||
| + | |||
| + | ==== Step 1: Running [[iPi Recorder]] ==== | ||
| + | |||
| + | Run [[iPi Recorder]] in a regular (default) mode used for recording action videos. Do not use darkening modes in "darkening for calibration" list (if available). Darkening is good for flashlight calibration, but bad for joints detection. | ||
| + | |||
| + | ==== Step 2: Record Calibration Video ==== | ||
| + | |||
| + | Start video recording. | ||
| + | |||
| + | Just walk around the whole capture area, and put hands up from time to time. | ||
| + | |||
| + | {{Tip|The exact movement is not so important, just try to cover the whole capture volume by arms and legs movement, or at least its perimeter.}} | ||
| + | |||
| + | {{Tip|You can use your action videos for calibration, if the body movement cover the whole capture area. Videos when you stand at the same place all the time will not allow for accurate calibration. }} | ||
| + | |||
| + | {{Tip|For successful calibration, 1000 frames shoud be enough. Recording long videos over 1 minute length does not make much sense, but you still can use them for calibration. }} | ||
| + | |||
| + | ==== Step 3: Take Note of Height of Your First Camera Over the Ground. ==== | ||
| + | |||
| + | Take note of height of your first camera over the ground. You will need this parameter later. | ||
| + | |||
| + | ==== Step 4: Process Calibration Video in [[iPi Mocap Studio]] ==== | ||
| + | |||
| + | {{Tip|You can postpone processing calibration video until after you finished recording other videos (e.g. your action videos). However, it is a good idea to process your calibration video as soon as it was recorded because it helps you ensure that you have good calibration.}} | ||
| + | |||
| + | To process calibration video please do the following: | ||
| + | * Create new calibration project in [[iPi Mocap Studio]]: | ||
| + | ** Press ''New'' button or select '''File > New Project''' menu item or use <tt>Ctrl+N</tt> (2) | ||
| + | *:[[Image:Calibration_Start_AI_1.png|center|800px]] | ||
| + | ** Choose '''Calibration project''' type in '''New Project Wizard'''. | ||
| + | ** Set the diagonal '''Field of View''' (FOV) for your cameras. If this screen is not shown, this means FOV is already known and you do not need to manually specify it. | ||
| + | *:{{Note|If you use Sony PS Eye or Logitech QuickCam 9000 cameras, leave the FOV value at the default 75 degrees.}} | ||
<center> | <center> | ||
{| | {| | ||
| − | |[[Image: | + | |[[Image:Select_Project_Type_Calibration.png|center|400px]] |
| − | |[[Image: | + | |[[Image:Calibration_Select_Camera_FOV.png|center|400px]] |
|} | |} | ||
</center> | </center> | ||
| + | * Adjust the [[Region-of-Interest]] to cover the part of video that contains movement around capture area. | ||
| + | * Normally, you need to use default calibration settings | ||
| + | ** Auto-detect initial camera positions turned on | ||
| + | *** If calibration fails to detect initial camera positions correctly, you can turn this checkbox off and set camera positions manually. But in most cases it will be easier to re-record calibration sequence. | ||
| + | ** Auto-adjust camera FOV turned off | ||
| + | *** Use this mode only if you did not know camera field of view requested during project creation (if it was requested) or if it was set by default (this may be recommended for particular camera models). | ||
| + | * Click '''Calibrate Based on AI Joints''' button and wait while the system finishes calibration (4). | ||
| + | |||
| + | ==== Complete Calibration ==== | ||
| + | |||
| + | Follow the same [[User_Guide_for_Multiple_PS_Eye_Cameras_Configuration#Step_6:_Check_Calibration_Quality|Steps 6-8]] described for flashlight calibration | ||
Latest revision as of 06:30, 31 October 2024
Contents
- 1 Flashlight Calibration
- 1.1 Importance of high frame rate
- 1.2 Glowing Marker
- 1.3 Step 1: Running iPi Recorder in Calibration Mode
- 1.4 Step 2: Record Calibration Video
- 1.5 Step 3: Stop Recording and Check Recorded Video
- 1.6 Step 4: Take Note of Height of Your First Camera Over the Ground.
- 1.7 Step 5: Process Calibration Video in iPi Mocap Studio
- 1.8 Step 6: Check Calibration Quality
- 1.9 Step 7: Check Ground Plane
- 1.10 Step 8: Set Scene Scale Using Camera Height as Reference
- 2 Calibration Based on AI Joints
Flashlight Calibration
Calibration is a process of computing accurate camera positions and orientations from a video of user waving a small glowing object called marker (for color/color+depth cameras). This step is essential and required for multi-camera system setup.
Importance of high frame rate
You should record calibration video at the same resolution as your action video and at the same (or higher) frame rate.
Calibration at a different resolution may lead to reduced accuracy because cameras usually have different minor distortions at different resolutions (caused by internal scaling algorithm).
Calibration at low frame rate may lead to reduced accuracy because of increased synchronization errors.
Glowing Marker
Mini Maglite flashlight is recommended for calibration. This is a very common flashlight in US and many other countries. Removing flashlight reflector converts it into an ideal glowing marker easily detectable by motion capture software.

If you cannot get a Mini Maglite, you can use some other similar flashlight.
Alternatively, you can use Sony Move motion controller with white light turned on.

Step 1: Running iPi Recorder in Calibration Mode
Run iPi Recorder and choose one of the darkening modes in "darkening for calibration" list (for Sony PS Eye cameras)

or set Exposure to reasonably small value (for DirectShow-compatible web cameras)

This is important because it helps to reduce motion blur during calibration.
Video will look dim in calibration mode.
Step 2: Record Calibration Video
Start video recording.
Move the marker slowly through your entire capture volume (front-top-right-bottom-left-back-top-right-bottom-left). Start from top and move the marker in a descending spiral motion.



Put the marker to the ground at each corner and at the center of capture volume. At least 4-5 ground points are needed for the correct detection of the groundplane.
Step 3: Stop Recording and Check Recorded Video
Check the video and make sure that:
- There is no significant motion blur (image of marker looks like a round spot rather than an ellipse or a luminescent line)
- Most of the time (80%-90% of all recording time) marker is visible in all cameras
Step 4: Take Note of Height of Your First Camera Over the Ground.
Take note of height of your first camera over the ground. You will need this parameter later.
Step 5: Process Calibration Video in iPi Mocap Studio
To process calibration video please do the following:
- Create new calibration project in iPi Mocap Studio:
- Press New button or select File > New Project menu item or use Ctrl+N (2)
- Choose Calibration project type in New Project Wizard.
- Set the diagonal Field of View (FOV) for your cameras. If this screen is not shown, this means FOV is already known and you do not need to manually specify it.
- Note: If you use Sony PS Eye or Logitech QuickCam 9000 cameras, leave the FOV value at the default 75 degrees.

- Adjust the Region-of-Interest to cover the part of video that contains the glowing marker (3).
- Change the size of light marker spot on video if needed.
- Medium usually work well, but you may need Large if you use Sony PS Move for calibration, or Small in some cases.
- Normally, you need to use default calibration settings
- Auto-detect initial camera positions turned on
- If calibration fails to detect initial camera positions correctly, you can turn this checkbox off and set camera positions manually. But in most cases it will be easier to re-record calibration sequence.
- Auto-adjust camera FOV turned off
- Use this mode only if you did not know camera field of view requested during project creation (if it was requested) or if it was set by default (this may be recommended for particular camera models).
- Auto-detect initial camera positions turned on
- Click Calibrate Based on Flashlight Marker button and wait while the system finishes calibration (4).
Step 6: Check Calibration Quality
Resulting scene should look like this:
Make sure you have Good or Perfect calibration result.
Step 7: Check Ground Plane
Ground plane should be detected automatically. Ground points are marked by yellow color.
- Make sure that ground points are actually near the ground plane.
- If ground plane is detected incorrectly, select ground points manually.
- Expand Manual Calibration Adjustment expander.
- First, unmark all ground points by pressing Clear all points button.
- For each ground point, click on it in 3D view and press Mark as ground button.
- You can cancel marking point as ground point by pressing Unmark ground button.
Step 8: Set Scene Scale Using Camera Height as Reference
Now cameras in your scene are properly oriented relative to other cameras and relative to ground plane. But you still need to find one more parameter: scene scale.
Use Сamera #1 height over ground to set correct scene scale.
- Save results to calibration project file or using Save scene... button on Scene tab (6).

Calibration Based on AI Joints
Novel AI-based pose estimation algorithms allow to use AI-detected joint positions instead of glowing object (flashlight), that makes the calibration workflow simpler.
Importance of high frame rate
You should record calibration video at the same resolution as your action video and at the same (or higher) frame rate.
Calibration at a different resolution may lead to reduced accuracy because cameras usually have different minor distortions at different resolutions (caused by internal scaling algorithm).
Calibration at low frame rate may lead to reduced accuracy because of increased synchronization errors.
Step 1: Running iPi Recorder
Run iPi Recorder in a regular (default) mode used for recording action videos. Do not use darkening modes in "darkening for calibration" list (if available). Darkening is good for flashlight calibration, but bad for joints detection.
Step 2: Record Calibration Video
Start video recording.
Just walk around the whole capture area, and put hands up from time to time.
Step 3: Take Note of Height of Your First Camera Over the Ground.
Take note of height of your first camera over the ground. You will need this parameter later.
Step 4: Process Calibration Video in iPi Mocap Studio
To process calibration video please do the following:
- Create new calibration project in iPi Mocap Studio:
- Press New button or select File > New Project menu item or use Ctrl+N (2)
- Choose Calibration project type in New Project Wizard.
- Set the diagonal Field of View (FOV) for your cameras. If this screen is not shown, this means FOV is already known and you do not need to manually specify it.
- Note: If you use Sony PS Eye or Logitech QuickCam 9000 cameras, leave the FOV value at the default 75 degrees.

- Adjust the Region-of-Interest to cover the part of video that contains movement around capture area.
- Normally, you need to use default calibration settings
- Auto-detect initial camera positions turned on
- If calibration fails to detect initial camera positions correctly, you can turn this checkbox off and set camera positions manually. But in most cases it will be easier to re-record calibration sequence.
- Auto-adjust camera FOV turned off
- Use this mode only if you did not know camera field of view requested during project creation (if it was requested) or if it was set by default (this may be recommended for particular camera models).
- Auto-detect initial camera positions turned on
- Click Calibrate Based on AI Joints button and wait while the system finishes calibration (4).
Complete Calibration
Follow the same Steps 6-8 described for flashlight calibration

