How to Use:

Make sure that two windows "Webcam Motion Capture" and "Webcam Motion Receiver" are opening.

Check if the webcam on your computer is ON and your upper body is shown on "Webcam Motion Capture" window.: - If your Webcam image is not shown on "Webcam Motion Capture" window, see the FAQ.

Move your hands and fingers, tilt your head, and blink!: - You will see that your avatar moves according to your move😊; - Also check the recommended setup!

Adjust "Body Filter" and "Face Filter": - Small value ⇨ fast reaction but noisy.; - Large value ⇨ slow reaction but smooth.; - (Tip) Set it to as large value as possible while the reaction is still fast enough.

Please also watch a great tutorial from Cooki Kunai!

Use Video as an Input

If your computer is not fast enough to perform smooth real-time tracking, you can try this approach!

First, shoot a video using your smartphone, etc. and send it to your computer. (You can also use this sample video for a quick test.)

On "Webcam Motion Capture", select "Video".

Select "New Video" as a source. Then, click "Open Video" to select your video.

Click "Output Folder" to specify the folder where processed video information is stored.

Click "Process".: - Once processing video is done, your processed video and control icons will appear.

Hit "play icon".: - You will see that your avatar moves according to your video!

Adjust "Body Filter" and "Face Filter": - Small value ⇨ fast reaction but noisy.; - Large value ⇨ slow reaction but smooth.; - (Tip) Set it to as large value as possible while the reaction is still fast enough.

Auto Expression Tracking

You can control the avatar's facial expression based on your own facial expressions captured by the webcam!

The avatar's facial expression is automatically set based on your facial expression and the registered facial expressions in the selected sample set^*.

* This feature is enabled by default. You can toggle between a fixed expression and Auto Expression Tracking using the keyboard.

Select either a Preset or Custom sample set from the dropdown menu (Figure (A)).

Each sample set includes:

Neutral Expression
2 Emotions Expressions

Select 2 from "FUN (RELAXED)", "ANGRY", "JOY (HAPPY)", "SORROW (SAD)", "SURPRISED", "EXTRA" using the corresponding dropdown menu (Figure (B)).

5 Lip Sync Expressions

"A", "I", "U", "E", "O"

For the Preset samples:: - The default registered facial expression is displayed for each corresponding avatar expression (Figure (C)).

For the Custom samples:

- You can register your custom facial expressions (Figure (C)) for each avatar's expression as follows.

Make sure that the webcam is enabled and click the corresponding "Update Samples" button (Figure (D)).
Your current expression is instantly reflected on the face model in real-time.
To register the expression, click the "Register" button.

- You can restore the default facial expression by clicking the "Reset" button (Figure (E)).

All facial expressions are recognized as deviations from the Neutral expression. Therefore, it is advisable to register your typical facial expression as the Neutral expression.

Use Animation

You can combine your favorite animation with Face and Hand Tracking to control your avatar. The loaded animation is applied to all the bones except the following bones.

When your hand is visible:

Head^*1 / Body Rotation Target^*1*2 / Upper Arms / Lower Arms / Hands / Fingers

When your hand is Not visible:

Head^*1 / Body Rotation Target^*1*2

*1: You can also apply animation to "Head" and "Body Rotation Target" by disabling the corresponding tracking from "Advanced Tracking Settings" in the Webcam Motion Receiver window.
*2: Body Rotation Target Bone can be set to either "Spine", "Chest", or "Upper Chest" from "Advanced Tracking Settings" in the Webcam Motion Receiver window.

Enable animation by checking "Use Animation" in the top-right corner of the "Webcam Motion Receiver" window.

You can select either "Preset" animation or "Custom" Animation.

For the Preset animation:

Select animation from the dropdown menu below.

For the custom animation:

You can load your own custom animation in the FBX file format from the "Load Animation" button. You can have your own custom animation either by:

Downloading your favorite animation from Mixamo for free. All the 2500+ animation data on Mixamo are royalty free even for commercial use!
Generating your custom animation using a Computer Graphics software such as Blender. Once generated, export the animation in the FBX file format.
- Please don't change the bone names while editing/exporting your avatar.
- You need to install VRM Add-on for Blender to edit your VRM avatar in Blender.

In addition to the main animation, you have the option to set up to five sub-animations. You can switch between them by using the assigned keys.

Set Up Mobile App

If you have an iPhone/iPad (iOS 13.0 or later), you can use a mobile app to enhance your avatar's facial expressions using Face Tracking! (For an Android phone, there is an alternative app.)

waidayo

Have a VRM model which is compatible with Perfect Sync.
- The default avatar is compatible with Perfect Sync so you can use it for a quick test. (You can download the avatar from here.)
Open "waidayo" on your iPhone/iPad.
Transfer your avatar to your iPhone/iPad by following this.
By following this, set up the connection between "waidayo" on your iPhone/iPad and "waidayo for PC" on your Windows/Mac.
- Once you confirm that the connection is established, you can close "waidayo for PC".
Then, open the Webcam Motion Capture, and on the "Webcam Motion Receiver" window, check "Track face using mobile app" as shown in this image.
Now, you will see that your avatar's facial expressions animate according to your facial expressions!
- Try puffing your cheeks, sticking out your tongue, etc.😊

Facemotion3d, iFacialMocap, MeowFace (Android)

Have a VRM model which is compatible with Perfect Sync.
- The default avatar is compatible with Perfect Sync so you can use it for a quick test. (You can download the avatar from here.)
Make sure that your computer and your phone are connecting to the same WiFi.
Open Facemotion3d/iFacialMocap/MeowFace on your Phone.
On the "Webcam Motion Receiver", check "Track face using mobile app" as shown in this image, then select the corresponding app from the drop-down menu below.
- The mobile app will automatically connect to the "Webcam Motion Receiver." If it does not automatically connect, enter your phone's IP address in the IP address under the drop-down menu and press the connect button^*.
Now, you will see that your avatar's facial expressions animate according to your facial expressions!
- Try puffing your cheeks, sticking out your tongue, etc.😊

* If you specify the IP address directly and it still does not connect, the network traffic may be blocked by a firewall on your PC.

Open "Windows Defender Firewall with Advanced Security" by pressing "Win + r", typing "wf.msc", and pressing Enter. Select "Inbound Rules" and find "webcam motion receiver.exe." By double clicking it, open "webcam motion receiver.exe Properties." Choose "Allow the connection" in the "Action" section (Refer to this image).

Table of Contents

Let's Get Started