Most decoration apps currently running on mobile phones such as Snow and Line Camera operate in the form of extracting facial landmarks and overlaying previously created objects. In addition, various image processing techniques such as beauty filters have recently been introduced to show more rich effects.
As AI technologies related to these visual effects, deep learning-based generative models such as GAN and VAE can be considered. Looking at the results published so far, it is possible to obtain results far exceeding the effects supported by the currently serviced app, but it has a disadvantage that it is much higher in complexity than facial landmark extraction. Because of this complexity problem, deep learning-based generative models are not only difficult to run directly on a mobile phone, but also difficult to increase the resolution of the applied image.
Server-based implementation rather than mobile phone alleviates this problem somewhat, but in this case, server cost increases rapidly as the number of users increases, and it is difficult to apply to video-based apps due to network delay.
HyperGAN is an open source GAN designed to run on mobile phones. HyperGAN does not solve all the problems mentioned above, and there are still many things to improve in terms of the quality of the result, but it seems that the reference code can sufficiently play a role. Here are the HyperGAN project page, the HyperGAN github repository with learning methods and model transformation methods, and a link to the HyperGAN android codebase:
