Difference between revisions of "How to Build Synced Multi-Camera Video"

Latest revision as of 12:29, 5 September 2023

Introduction

To process video recorded by action cameras (or other standalone camcorders) in iPi Mocap Studio you have to create multi-camera .iPiVideo file from separate MP4/MOV/AVI/MPEG video files recorded by these cameras:

Preparations

First of all, copy MP4/MOV/AVI/MPEG video files from each camera to computer.

There are two hints here:

1. It's better to use microSD-card reader to copy files because it provides much higher speed of this operation in comparison with USB-connection.
2. To distinguish files from different cameras it is convenient to create a sub-folder for each camera. And name of such sub-folder should contain label/nickname of corresponding camera.
   For example: if you have cameras with labels/names "cam1", "cam2", "cam3" and "cam4" then you can create the following structure of folders and files on your local disk:

YYYY-MM-DD-session-name
   cam1
      video files from "cam1" camera
   cam2
      video files from "cam2" camera
   cam3
      video files from "cam3" camera
   cam4
      video files from "cam4" camera

Alternatively you can rename video files right after copying (or even during copying) to local disk while you still remember which camera has produced those files. Such approach is used in the following sample: Sample Videos and Projects (four GoPro).

Overall Procedure

• Run iPi Recorder and click Build Combined Video button in the toolbar at the home screen.

• For each camera:
  • add video from this camera by clicking Add Video button in the toolbar — see mark (1) on screenshot below
  • choose this camera by clicking on the image from this camera — (2)
  • in the left panel (3) set camera parameters by clicking on Load button in camera parameters section — (4)
    • choose appropriate .iPiCam file you saved for this camera during camera optics calibration
• Now videos from all cameras are added.
• Define synchronization between cameras:
  • automatically:
    • if you're using flashes as sync events then simply click Sync by Flashes button on toolbar — (E)
    • and check correctness of automatic synchronization
  • or manually:
    • for each camera, click button Edit in sync frame section of the left panel (3) and choose a frame with synchronization event — (5)
  • for details see below
• Now all required parameters (camera optics parameters and synchronization frames) are defined for all source videos.

• Evaluate background:
  • using markers (6) and (7) select an interval on the timeline where there is no actor and moving objects (the length of this interval should be about a couple of seconds)
  • alternatively, to set an interval on the timeline you can use controls in section INTERVAL of the toolbar — (8)
  • click Evaluate Background button in the toolbar to evaluate background for all cameras simultaneously from the selected interval on the timeline — (9)
  • alternatively:
    • you can evaluate background for each camera separately using button Edit... in section background in the left panel (3) — (A)
    • or you can even load background from file if you've already evaluated it before from different video — (B)
    • for details see below
• Now everything is ready to be saved into destination .iPiVideo file:
  • using markers (6) and (7) on the timeline or controls (8) in the toolbar, select an interval you want to save to a resulting video
    • for a calibration video it could be an interval where the glowing marker is visible
    • for an action video it could be an interval with needed action sequence or movement
    • or you can save the whole video if you're not sure which exact interval will be used for processing in iPi Mocap Studio
  • click Save Combined Video button in the toolbar and choose destination file name — (C)
  • also before saving you can set compression quality for destination video — (D)
    • but in most cases the default value (70%) works very well providing a good balance between image quality and size of the resulting video
• Finally you can open the resulting .iPiVideo file in iPi Mocap Studio and process it.

Important Aspects

Now let's discuss some aspects in details:

Cameras and Their Parameters

1. It is strictly mandatory to add cameras in one and the same order for scene calibration and action videos. In other case results of scene calibration will not match action video(s). By the way, you can use drag-and-drop to reorder cameras in the builder.
2. Don't forget to apply camera parameters by loading them from appropriate .iPiCam file. Remember that this .iPiCam file must be for this specific camera and for exactly the same resolution and view/FOV settings (FPS may differ). The thing is that camera optics parameters significantly vary from item to item (even when all items are of the same make and model) and depends on resolution and view/FOV settings.
3. You can adjust the scale of undistort filter right in Combined Video Builder:

Be aware that the same scale value must be used for scene calibration and action videos (per camera, of course). For this reason use this feature with a great care.

Synchronization

To build a combined video, source videos from individual cameras must be synchronized:

To do it, sync frame must be defined for each camera. Sync frame is an frame with some short and sharp visual event. For details see How to Record Video with Action Cameras#Synchronization Event.

Example of synchronization using an external powerful on-camera flash:

	Prev. frame (before sync)	Sync frame	Next frame (after sync)
cam1
cam2
cam3
cam4

Starting from version 4.2.0 of iPi Recorder, Combined Video Builder supports two approaches to synchronization:

1. fully automatic synchronization based on detected illumination produced by an external powerful on-camera flash
2. manual synchronization based on any short/sharp visual event visible in all cameras simultaneously

How to synchronize automatically

• Just click Sync by Flash button in toolbar.
• In case of any troubles with automatic synchronization, you'll get appropriate message.
• In case of success, it is recommended to check synchronization (visually, approx.) to be on safe side.
• Also, you can see all detected flashes and selected sync frame for each camera on SYNC FRAME tab (click Edit... button in sync frame section of the left panel to call this tab):

On this screenshot:

• (1) and (2) — detected flashes
• (3) — sync frames
• (4) — buttons for jumping to next/prev nearest frame with flash detected
• (5) — button to run automatic flash detector (for .iPiVideo files converted from regular videos like avi/mp4/mov, this detection takes place during conversion procedure).

How to synchronize manually

For each camera:

• Select a camera and click Edit... button in sync frame section of the left panel — see mark (1) on the screenshot below.
• Additional tab SYNC FRAME opens containing only video from selected camera.
• Using the timeline (2) choose a frame with synchronization event.
  • it is convenient to use buttons Previous Frame and Next Frame to position precisely on required frame — see marks (3) and (4) on the screenshot below
• After that click Set to Current Frame button in the toolbar — (5).
• Now switch back to BUILD tab by clicking on it — (6).
• Also, you can adjust synchronization frame right in the builder using buttons (7) and (8).
• Alternatively, you can set a synchronization frame to the current frame shown in the builder using button Current Frame — (9).

Background

And the last but not the least thing in the way to combine video is background evaluation. Background is an image of clear scene without actors and moving objects:

There are three ways to evaluate the background:

1. Simultaneously for all the cameras
   • this method is the easiest and described above in section #Overall Procedure
   • but it requires that clear scene is simultaneously visible in all cameras which can be problematic to achieve in real life conditions
2. For each camera (source video) separately
   • select required camera and click Edit... button in background section of the left panel — see mark (1) on the screenshot below
   • additional tab BACKGROUND opens with a video from selected camera only
   • using triangles (2) on the timeline or controls (3) in the toolbar, select an interval on the timeline where there is no actor and moving objects (the length of this interval should be about a couple of seconds)
   • after that click Evaluate Background button in the toolbar — (4)
   • optionally you can save the result (evaluated background) to a file on disk to use later with different videos (see below) — (5)
   • now switch back to BUILD tab by clicking on it — (6)

   

3. Or you can evaluate background based on other video, save it to a file and then just load it when you need the background for appropriate camera.

In any case it is worth to check the background quality after evaluation using Hide Background button in the toolbar of Combined Video Builder:

Sample

You can use the following sample videos to train in this subject without the need to record your own videos: Sample Videos and Projects (four GoPro).