FrankMocap, a technology released by Facebook AI Research (FAIR), is responsible for extracting a pose for a 3D model from a single image or video. In particular, it can be said that it can estimate not only the body but also the shape of the hand.
In general, motion capture technology is used to extract 3D poses, but it is also a very expensive operation when creating content or games because it requires repetitive performance of a model with a marker and specialized equipment consisting of multiple cameras. Of course, the data extracted with FrankMocap is not at a level that can be applied directly to the final product, but it can be said that it is highly likely to become a service in that it can greatly reduce the cost of initial work. The Github link is below:
Here are some demos from the github repository:
- Body Motion Capture:
- Hand Motion Capture
- Egocentric Hand Motion Capture
- Whole body Motion Capture (body + hands)
FrankMocap's results are quite good among similar studies, and if it is properly post-processed, it is likely to be quite useful. In the future, we plan to conduct a technical evaluation and applicability review process.
