How to Use:

Make sure that two windows "Webcam Motion Capture" and "Webcam Motion Receiver" are opening.

Check if the webcam on your computer is ON and your upper body is shown on "Webcam Motion Capture" window.: - If your Webcam image is not shown on "Webcam Motion Capture" window, see the FAQ.

Move your hands and fingers, tilt your head, and blink!: - You will see that your avatar moves according to your move😊; - Also check the recommended setup!

Adjust "Body Filter" and "Face Filter": - Small value ⇨ fast reaction but noisy.; - Large value ⇨ slow reaction but smooth.; - (Tip) Set it to as large value as possible while the reaction is still fast enough.

Please also watch a great tutorial from Cooki Kunai!

Use Video as an Input

If your computer is not fast enough to perform smooth real-time tracking, you can try this approach!

First, shoot a video using your smartphone, etc. and send it to your computer. (You can also use this sample video for a quick test.)

On "Webcam Motion Capture", select "Video".

Select "New Video" as a source. Then, click "Open Video" to select your video.

Click "Output Folder" to specify the folder where processed video information is stored.

Click "Process".: - Once processing video is done, your processed video and control icons will appear.

Hit "play icon".: - You will see that your avatar moves according to your video!

Adjust "Body Filter" and "Face Filter": - Small value ⇨ fast reaction but noisy.; - Large value ⇨ slow reaction but smooth.; - (Tip) Set it to as large value as possible while the reaction is still fast enough.

Use Animation

You can combine your favorite animation with Face and Hand Tracking to control your avatar. The loaded animation is applied to all the bones except the following bones.

When your hand is visible:

Head^*1 / Body Rotation Target^*1*2 / Upper Arms / Lower Arms / Hands / Fingers

When your hand is Not visible:

Head^*1 / Body Rotation Target^*1*2

*1: You can also apply animation to "Head" and "Body Rotation Target" by disabling the corresponding tracking from "Advanced Tracking Settings" in the Webcam Motion Receiver window.
*2: Body Rotation Target Bone can be set to either "Spine", "Chest", or "Upper Chest" from "Advanced Tracking Settings" in the Webcam Motion Receiver window.

Enable animation by checking "Use Animation" in the top-right corner of the "Webcam Motion Receiver" window.

You can select either "Preset" animation or "Custom" Animation.

For the Preset animation:

Select animation from the dropdown menu below.

For the custom animation:

You can load your own custom animation in the FBX file format from the "Load Animation" button. You can have your own custom animation either by:

Downloading your favorite animation from Mixamo for free. All the 2500+ animation data on Mixamo are royalty free even for commercial use!
Generating your custom animation using a Computer Graphics software such as Blender. Once generated, export the animation in the FBX file format.
- Please don't change the bone names while editing/exporting your avatar.
- You need to install VRM Add-on for Blender to edit your VRM avatar in Blender.

In addition to the main animation, you have the option to set up to five sub-animations. You can switch between them by using the assigned keys.

Set Up Mobile App

If you have an iPhone/iPad (iOS 13.0 or later), you can use a mobile app to enhance your avatar's facial expressions using Face Tracking! (For an Android phone, there is an alternative app.)

waidayo

Have a VRM model which is compatible with Perfect Sync.
- The default avatar is compatible with Perfect Sync so you can use it for a quick test. (You can download the avatar from here.)
Open "waidayo" on your iPhone/iPad.
Transfer your avatar to your iPhone/iPad by following this.
By following this, set up the connection between "waidayo" on your iPhone/iPad and "waidayo for PC" on your Windows/Mac.
- Once you confirm that the connection is established, you can close "waidayo for PC".
Then, open the Webcam Motion Capture, and on the "Webcam Motion Receiver" window, check "Track face using mobile app" as shown in this image.
Now, you will see that your avatar's facial expressions animate according to your facial expressions!
- Try puffing your cheeks, sticking out your tongue, etc.😊

Facemotion3d, iFacialMocap, MeowFace (Android)

Have a VRM model which is compatible with Perfect Sync.
- The default avatar is compatible with Perfect Sync so you can use it for a quick test. (You can download the avatar from here.)
Make sure that your computer and your phone are connecting to the same WiFi.
Open Facemotion3d/iFacialMocap/MeowFace on your Phone.
On the "Webcam Motion Receiver", check "Track face using mobile app" as shown in this image, then select the corresponding app from the drop-down menu below.
- The mobile app will automatically connect to the "Webcam Motion Receiver." If it does not automatically connect, enter your phone's IP address in the IP address under the drop-down menu and press the connect button^*.
Now, you will see that your avatar's facial expressions animate according to your facial expressions!
- Try puffing your cheeks, sticking out your tongue, etc.😊

* If you specify the IP address directly and it still does not connect, the network traffic may be blocked by a firewall on your PC.

Open "Windows Defender Firewall with Advanced Security" by pressing "Win + r", typing "wf.msc", and pressing Enter. Select "Inbound Rules" and find "webcam motion receiver.exe." By double clicking it, open "webcam motion receiver.exe Properties." Choose "Allow the connection" in the "Action" section (Refer to this image).

Table of Contents