LipGAN is a technology that generates the shape of the lips of a face image using a voice signal, and when it is actually applied to a video, it was somewhat disappointing in terms of visual artifacts and the naturalness of movement. To improve this, the discriminator is not a single frame, but a plurality of consecutive…
I've heard stories that you can know what you're talking about with just the movements of your lips if you get special training, but the research in the link was realized with AI.
LipGan is the study of creating mouth shapes from speech signals. It is a technique that can be useful for creating an animation of a virtual character's mouth, but when applied in practice, the limitation is clear because only the lips of a character standing still move. In fact, humans...
