Skip to content

Input video instead of image for inference. #45

Description

@loic-combis

Hi there,

I'm trying to test style avatar to correct the lip sync generation from Wav2Lip.

(I currently have an issue with the preprocessing in Faceverse LizhenWangT/FaceVerse#36)

But regardless, we aim to do the following:

  • (1) Train Style avatar on the original video/face.
  • (2) Run Wav2Lip on the original video with a new audio, to get a new video with but poor quality lip sync.
  • Question for this step: Can we use the fine-tuned style avatar model (from step 1), to correct the lip movement on the video generated step 2?

Note that we try to avoid inputing a single image and recreate the whole speech on the avatar, and rather trying to simply correct the video.

Thanks for your help and your work!!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions