How to Build Synced Multi-Camera Video - Revision history

Andrew: /* Synchronization */

2023-09-05T16:29:41Z

‎Synchronization

Andrew: /* How to synchronize automatically */

2019-02-08T16:53:57Z

‎How to synchronize automatically

Andrew: /* Synchronization */

2019-02-06T17:28:38Z

‎Synchronization

Andrew: /* Overall Procedure */

2019-02-06T16:35:20Z

‎Overall Procedure

Andrew: /* Overall Procedure */

2019-02-06T13:37:12Z

‎Overall Procedure

Vmaslov: /* Sample */

2018-08-15T20:47:28Z

‎Sample

Vmaslov: /* Background */

2018-08-15T20:47:06Z

‎Background

Vmaslov: /* Synchronization */

2018-08-15T18:34:06Z

‎Synchronization

Vmaslov: /* Cameras and Their Parameters */

2018-08-15T18:21:27Z

‎Cameras and Their Parameters

Vmaslov: /* Overall Procedure */

2018-08-15T18:06:56Z

‎Overall Procedure

@@ Line 75: / Line 75: @@
 To build a combined video, source videos from individual cameras must be synchronized:
-<center>[[File:build-combined-video-sync-schema.png|1000px|link=]]</center>\
+<center>[[File:build-combined-video-sync-schema.png|1000px|link=]]</center>
 To do it, '''sync frame''' must be defined for each camera. '''Sync frame''' is an frame with some short and sharp visual event. For details see [[How to Record Video with Action Cameras#Synchronization Event]].

@@ Line 96: / Line 96: @@
 * Just click '''Sync by Flash''' button in toolbar.
 * In case of any troubles with automatic synchronization, you'll get appropriate message.
-* In case of success, it is recommended to check synchronization (visually, by eye) to be on safe side.
+* In case of success, it is recommended to check synchronization (visually, approx.) to be on safe side.
 * Also, you can see all detected flashes and selected '''sync frame''' for each camera on '''SYNC FRAME''' tab (click '''Edit...''' button in '''sync frame''' section of the left panel to call this tab):

@@ Line 74: / Line 74: @@
 === Synchronization ===
-To build a combined video, source videos from individual cameras must be synchronized. Current version of '''Combined Video Builder''' supports only manual synchronization based on some short/sharp visual event visible in all cameras simultaneously. A frame where such event takes place is called a '''sync frame'''. That is, you have to set a sync frame for each source video:
+To build a combined video, source videos from individual cameras must be synchronized:
-<center>[[File:build-combined-video-sync-schema.png|1000px|link=]]</center>
+To do it, '''sync frame''' must be defined for each camera. '''Sync frame''' is an frame with some short and sharp visual event. For details see [[How to Record Video with Action Cameras#Synchronization Event]].
-We recommend to use the following visual events for synchronization:
+Example of synchronization using an external powerful on-camera flash:
-* the moment when you turn on or off a flash light (or [https://www.amazon.com/Playstation-3-Move-Motion-Controller/dp/B002I0J51U/ref=as_sl_pc_qf_sp_asin_til?tag=wwwipisoftcom-20&linkCode=w00&linkId=3LAZCYWUQQFLH2LX&creativeASIN=B002I0J51U Sony Move])
-* manual firing of an external on-camera flash
-−
-For details see [[How to Record Video with Action Cameras#Synchronization Event]].
-−
-Example of synchronization using an external on-camera flash in manual mode:
 <center><table border="0">
 <tr><td></td><td align="center">'''Prev. frame'''<br/>(before sync)</td><td align="center">'''Sync frame'''</td><td align="center">'''Next frame'''<br/>(after sync)</td></tr>
@@ Line 94: / Line 89: @@
 <center>''(In the 3rd camera the flash is not visible directly, but you can see illumination it produces.)''</center>
-How to set synchronization frame:
+Starting from version 4.2.0 of [[iPi Recorder]], '''Combined Video Builder''' supports two approaches to synchronization:
 * Select a camera and click '''Edit...''' button in '''sync frame''' section of the left panel — see mark '''(1)''' on the screenshot below.
 * Additional tab '''SYNC FRAME''' opens containing only video from selected camera.

@@ Line 35: / Line 35: @@
 * Define synchronization between cameras:
 ** automatically:
-*** if you're using flashes as sync events then simply click '''Sync by Flashes''' button on toolbar
+*** if you're using flashes as sync events then simply click '''Sync by Flashes''' button on toolbar — '''(E)'''
 *** and check correctness of automatic synchronization
 ** or manually:

@@ Line 32: / Line 32: @@
 ** in the left panel '''(3)''' set camera parameters by clicking on '''Load''' button in '''camera parameters''' section — '''(4)'''
 *** choose appropriate '''.iPiCam''' file you saved for this camera during [[How to Calibrate Camera Optics|camera optics calibration]]
-** click button '''Edit''' in '''sync frame''' section of the left panel '''(3)''' and choose a frame with synchronization event — '''(5)'''
+* Now videos from all cameras are added.
-*** for details see [[#Synchronization|below]]
+* Define synchronization between cameras:
-* Now videos from all cameras are added and required parameters (camera parameters and synchronization frames) are defined for them.
+** automatically:
 <center>[[File:build-combined-video-workflow.png|900px|link=]]</center>
 * Evaluate background:

@@ Line 125: / Line 125: @@
 == Sample ==
-You can use the following sample videos to train in this subject without need to record your own videos:
+You can use the following sample videos to train in this subject without the need to record your own videos:
 [[Sample Videos and Projects (four GoPro)]].

@@ Line 102: / Line 102: @@
 === Background ===
-And the last but not the list thing in the way to combined video is background evaluation. Background is an image of clear scene without actors and moving objects:
+And the last but not the least thing in the way to combine video is background evaluation. Background is an image of clear scene without actors and moving objects:
 <center>[[File:action-cameras-background-yes-and-no.jpg|1000px|link=]]</center>
-There are three ways to evaluate background:
+There are three ways to evaluate the background:
-# Simultaneously for all cameras
+# Simultaneously for all the cameras
 #* this method is the easiest and described above in section [[#Overall Procedure]]
 #* but it requires that clear scene is simultaneously visible in all cameras which can be problematic to achieve in real life conditions
 # For each camera (source video) separately
-#* select required camera and click '''Edit...''' button in '''background''' section of left panel — see mark '''(1)''' on screenshot below
+#* select required camera and click '''Edit...''' button in '''background''' section of the left panel — see mark '''(1)''' on the screenshot below
-#* additional tab '''BACKGROUND''' opens with single video in it
+#* additional tab '''BACKGROUND''' opens with a video from selected camera only
-#* using triangles '''(2)''' on timeline or controls '''(3)''' in toolbar, select interval on timeline where there is no actor and moving objects (the length of this interval should be about a couple of seconds)
+#* using triangles '''(2)''' on the timeline or controls '''(3)''' in the toolbar, select an interval on the timeline where there is no actor and moving objects (the length of this interval should be about a couple of seconds)
-#* after that click '''Evaluate Background''' button in toolbar — '''(4)'''
+#* after that click '''Evaluate Background''' button in the toolbar — '''(4)'''
-#* optionally you can save result (evaluated background) to file on disk to use later with different videos (see below) — '''(5)'''
+#* optionally you can save the result (evaluated background) to a file on disk to use later with different videos (see below) — '''(5)'''
 #* now switch back to '''BUILD''' tab by clicking on it — '''(6)'''
 #:<center>[[File:build-combined-video-how-to-evaluate-background.jpg|1000px|link=]]</center>
-# Or you can evaluate background based on other video, save it to file and then just load it when you need background for appropriate camera.
+# Or you can evaluate background based on other video, save it to a file and then just load it when you need the background for appropriate camera.
-In any case it is worth to check background quality after evaluation using '''Hide Background''' button in toolbar of '''Combined Video Builder''':
+In any case it is worth to check the background quality after evaluation using '''Hide Background''' button in the toolbar of '''Combined Video Builder''':
 <center>[[File:build-combined-video-hide-background.jpg|753px|link=]]</center>
 <center>''(In this particular case background subtraction is not ideal but good enough for motion capture needs.)''</center>

@@ Line 68: / Line 68: @@
 === Synchronization ===
-To build combined video source videos from separate cameras must be synchronized. Current version of '''Combined Video Builder''' supports only manual synchronization based on some short/sharp visual event visible in all cameras simultaneously. Frame where such event takes place is called as '''sync frame'''. That is, you have to set sync frame for each source video:
+To build a combined video, source videos from individual cameras must be synchronized. Current version of '''Combined Video Builder''' supports only manual synchronization based on some short/sharp visual event visible in all cameras simultaneously. A frame where such event takes place is called a '''sync frame'''. That is, you have to set a sync frame for each source video:
 <center>[[File:build-combined-video-sync-schema.png|1000px|link=]]</center>
 We recommend to use the following visual events for synchronization:
-* the moment when you turn on/off flash light (or [https://www.amazon.com/Playstation-3-Move-Motion-Controller/dp/B002I0J51U/ref=as_sl_pc_qf_sp_asin_til?tag=wwwipisoftcom-20&linkCode=w00&linkId=3LAZCYWUQQFLH2LX&creativeASIN=B002I0J51U Sony Move])
+* the moment when you turn on or off a flash light (or [https://www.amazon.com/Playstation-3-Move-Motion-Controller/dp/B002I0J51U/ref=as_sl_pc_qf_sp_asin_til?tag=wwwipisoftcom-20&linkCode=w00&linkId=3LAZCYWUQQFLH2LX&creativeASIN=B002I0J51U Sony Move])
-* manual firing of external on-camera flash
+* manual firing of an external on-camera flash
 For details see [[How to Record Video with Action Cameras#Synchronization Event]].
-Example of synchronization using external on-camera flash in manual mode:
+Example of synchronization using an external on-camera flash in manual mode:
 <center><table border="0">
 <tr><td></td><td align="center">'''Prev. frame'''<br/>(before sync)</td><td align="center">'''Sync frame'''</td><td align="center">'''Next frame'''<br/>(after sync)</td></tr>
@@ Line 86: / Line 86: @@
 <tr><td>cam4&nbsp;</td><td>[[File:frame-before-sync-cam4.jpg|320px|link=]]</td><td>[[File:frame-sync-cam4.jpg|320px|link=]]</td><td>[[File:frame-after-sync-cam4.jpg|320px|link=]]</td></tr>
 </table></center>
-<center>''(In the 3rd camera flash is not directly visible, but you can see illumination produced by this flash.)''</center>
+<center>''(In the 3rd camera the flash is not visible directly, but you can see illumination it produces.)''</center>
 How to set synchronization frame:
-* Select required camera and click '''Edit...''' button in '''sync frame''' section of left panel — see mark '''(1)''' on screenshot below.
+* Select a camera and click '''Edit...''' button in '''sync frame''' section of the left panel — see mark '''(1)''' on the screenshot below.
-* Additional tab '''SYNC FRAME''' opens with single video in it.
+* Additional tab '''SYNC FRAME''' opens containing only video from selected camera.
-* Using timeline '''(2)''' choose frame with synchronization event.
+* Using the timeline '''(2)''' choose a frame with synchronization event.
-** it is convenient to use buttons '''Previous Frame''' and '''Next Frame''' to position precisely on required frame — see marks '''(3)''' and '''(4)''' on screenshot below
+** it is convenient to use buttons '''Previous Frame''' and '''Next Frame''' to position precisely on required frame — see marks '''(3)''' and '''(4)''' on the screenshot below
-* After that click '''Set to Current Frame''' button in toolbar — '''(5)'''.
+* After that click '''Set to Current Frame''' button in the toolbar — '''(5)'''.
 * Now switch back to '''BUILD''' tab by clicking on it — '''(6)'''.
-* Also, you can adjust synchronization frame right in builder using buttons '''(7)''' and '''(8)'''.
+* Also, you can adjust synchronization frame right in the builder using buttons '''(7)''' and '''(8)'''.
-* Alternatively, you can set synchronization frame to current frame visible in '''BUILDER''' using button '''Current Frame''' — '''(9)'''.
+* Alternatively, you can set a synchronization frame to the current frame shown in the builder using button '''Current Frame''' — '''(9)'''.
 <center>[[File:build-combined-video-how-to-set-sync-frame.jpg|1000px|link=]]</center>

@@ Line 60: / Line 60: @@
 === Cameras and Their Parameters ===
-# It is strictly mandatory to add cameras in one and the same order for scene calibration and action videos. In other case results of scene calibration will not match action video(s). By the way, you can use drag-and-drop to reorder cameras.
+# It is strictly mandatory to add cameras in one and the same order for scene calibration and action videos. In other case results of scene calibration will not match action video(s). By the way, you can use drag-and-drop to reorder cameras in the builder.
-# Don't forget to apply camera parameters by loading them from appropriate '''.iPiCam''' file. Remember that this '''.iPiCam''' file must be for exactly the same camera item and for exactly the same resolution and view/FOV settings (FPS may differ). The thing is that camera optics parameters significantly vary from item to item (even when all items are of one and the same model) and depends on resolution and view/FOV settings.
+# Don't forget to apply camera parameters by loading them from appropriate '''.iPiCam''' file. Remember that this '''.iPiCam''' file must be for this specific camera and for exactly the same resolution and view/FOV settings (FPS may differ). The thing is that camera optics parameters significantly vary from item to item (even when all items are of the same make and model) and depends on resolution and view/FOV settings.
-# You can adjust [[How to Calibrate Camera Optics#Undistorting and Scale|scale of undistort filter]] right in '''Combined Video Builder''':
+# You can adjust [[How to Calibrate Camera Optics#Undistorting and Scale|the scale of undistort filter]] right in '''Combined Video Builder''':
 <center>[[File:build-combined-video-scale.png|300px|link=]]</center>
-:: But be aware that this scale must be one and the same for scene calibration and action videos (per camera, of course). For this reason use this feature with great care.
+:: Be aware that the same scale value must be used for scene calibration and action videos (per camera, of course). For this reason use this feature with a great care.
 === Synchronization ===

@@ Line 28: / Line 28: @@
 <center>[[File:build-combined-video-tool-button.png|500px|link=]]</center>
 * For each camera:
-** add video from this camera by clicking '''Add Video''' button in toolbar — see mark '''(1)''' on screenshot below
+** add video from this camera by clicking '''Add Video''' button in the toolbar — see mark '''(1)''' on screenshot below
-** choose this camera by clicking on image from this camera — '''(2)'''
+** choose this camera by clicking on the image from this camera — '''(2)'''
 ** in the left panel '''(3)''' set camera parameters by clicking on '''Load''' button in '''camera parameters''' section — '''(4)'''
 *** choose appropriate '''.iPiCam''' file you saved for this camera during [[How to Calibrate Camera Optics|camera optics calibration]]
-** click button '''Edit''' in '''sync frame''' section of left panel '''(3)''' and choose frame with synchronization event — '''(5)'''
+** click button '''Edit''' in '''sync frame''' section of the left panel '''(3)''' and choose a frame with synchronization event — '''(5)'''
 *** for details see [[#Synchronization|below]]
 * Now videos from all cameras are added and required parameters (camera parameters and synchronization frames) are defined for them.
 <center>[[File:build-combined-video-workflow.png|900px|link=]]</center>
 * Evaluate background:
-** using markers '''(6)''' and '''(7)''' select interval on timeline where there is no actor and moving objects (the length of this interval should be about a couple of seconds)
+** using markers '''(6)''' and '''(7)''' select an interval on the timeline where there is no actor and moving objects (the length of this interval should be about a couple of seconds)
-** alternatively, to set interval on timeline you can use controls in section '''INTERVAL''' of toolbar — '''(8)'''
+** alternatively, to set an interval on the timeline you can use controls in section '''INTERVAL''' of the toolbar — '''(8)'''
-** click '''Evaluate Background''' button in toolbar to evaluate background for all cameras simultaneously from selected interval on timeline — '''(9)'''
+** click '''Evaluate Background''' button in the toolbar to evaluate background for all cameras simultaneously from the selected interval on the timeline — '''(9)'''
 ** alternatively:
-*** you can evaluate background for each camera separately using button '''Edit...''' in section '''background''' in left panel '''(3)''' — '''(A)'''
+*** you can evaluate background for each camera separately using button '''Edit...''' in section '''background''' in the left panel '''(3)''' — '''(A)'''
 *** or you can even load background from file if you've already evaluated it before from different video — '''(B)'''
 *** for details see [[#Background|below]]
 * Now everything is ready to be saved into destination [[.iPiVideo File Extension|.iPiVideo]] file:
-** using markers '''(6)''' and '''(7)''' on timeline or controls '''(8)''' in toolbar, select fragment you want to save to result video
+** using markers '''(6)''' and '''(7)''' on the timeline or controls '''(8)''' in the toolbar, select an interval you want to save to a resulting video
-*** for calibration video it could be fragment where glowing marker is visible
+*** for a calibration video it could be an interval where the glowing marker is visible
-*** for action video it could be fragment with required action sequence or movement
+*** for an action video it could be an interval with needed action sequence or movement
-*** or you can save the whole video if you're not sure what fragment exactly will be used for processing in [[iPi Mocap Studio]]
+*** or you can save the whole video if you're not sure which exact interval will be used for processing in [[iPi Mocap Studio]]
-** click '''Save Combined Video''' button in toolbar and choose destination file name — '''(C)'''
+** click '''Save Combined Video''' button in the toolbar and choose destination file name — '''(C)'''
 ** also before saving you can set compression quality for destination video — '''(D)'''
-*** but in most cases default value (70%) works very well providing good balance between image quality in result video and size of this video
+*** but in most cases the default value (70%) works very well providing a good balance between image quality and size of the resulting video
-* After all you can open result [[.iPiVideo File Extension|.iPiVideo]] file in [[iPi Mocap Studio]] and process it.
+* Finally you can open the resulting [[.iPiVideo File Extension|.iPiVideo]] file in [[iPi Mocap Studio]] and process it.
 == Important Aspects ==