Tensorflow Lite is a software package that contains tools that allow AI models trained with Tensorflow to run on mobile devices. It is said to be running on over 4 billion devices now. Basically, since the trained model is converted to Tensorflow Lite format and used on a mobile device, it has the advantage of using the existing models as it is.
The results of many AI projects require high-performance GPUs and are made under the assumption that they operate in a PC environment, so there are many cases where a separate inference server is set up and accessed through API for service. However, in this case, the infrastructure cost increases proportionally as the number of users increases, and there is a problem that fundamental network latency is required. There is also an issue of utilization of personal information. This problem can be avoided by making inferences about the AI model directly on the user's device, and the utilization of on-device AI is expected to continue to increase, with a separate AI inference accelerator installed on smartphones recently.
A site called Awesome Tensorflow Lite is a collection of various mobile app sample projects using Tensorflow Lite. The following is a collection of projects organized in the awesome tensorflow lite github repository. There are projects in various fields such as compute vision, text, speech, and recommendation.
Computer vision
Task | Model | App | Reference | Source |
---|---|---|---|
Classification | MobileNetV1 (download) | Android | iOS | Raspberry Pi | Overview | tensorflow.org |
Classification | MobileNetV2 | Recognize Flowers on Android Codelab | Android | TensorFlow team |
Classification | MobileNetV2 | Skin Lesion Detection Android | Community |
Classification | EfficientNet-Lite0 (download) | Icon Classifier Colab & Android | tutorial 1 | tutorial 2 | Community |
Object detection | Quantized COCO SSD MobileNet v1 (download) | Android | iOS | Overview | tensorflow.org |
Object detection | YOLO | Flutter | Paper | Community |
Object detection | MobileNetV2 SSD (download) | Reference | MediaPipe |
Object detection | MobileDet (Paper) | Blog post (includes the TFLite conversion process) | MobileDet is from University of Wisconsin-Madison and Google and the blog post is from the Community |
License Plate detection | SSD MobileNet (download) | Flutter | Community |
Face detection | BlazeFace (download) | Paper | MediaPipe |
Hand detection & tracking | Palm detection & hand landmarks (download) | Blog post | Model card | MediaPipe |
Pose estimation | Posenet (download) | Android | iOS | Overview | tensorflow.org |
Segmentation | DeepLab V3 (download) | Android & iOS | Overview | Flutter Image | Realtime | Paper | tf.org & Community |
Segmentation | Different variants of DeepLab V3 models | Models on TF Hub with Colab Notebooks | Community |
Hair Segmentation | Download | Paper | MediaPipe |
Style transfer | Arbitrary image stylization | Overview | Android | Flutter | tf.org & Community |
Style transfer | Better-quality style transfer models in .tflite | Models on TF Hub with Colab Notebooks | Community |
GANs | U-GAT-IT (Selfie2Anime) | Project repo | Android | Tutorial | Community |
GANs | White-box CartoonGAN (download) | Project repo | Android | Tutorial | Community |
Video Style Transfer | Download: Dynamic range models) | Android | Tutorial | Community |
Segmentation & Style transfer | DeepLabV3 & Style Transfer models | Project repo | Android | Tutorial | Community |
Low-light image enhancement | Models on TF Hub | Project repo | Original Paper | | Community |
Text Detection | CRAFT Text Detector (Paper) | Download | Project Repository | Blog1-Conversion to TFLite | Blog2-EAST vs CRAFT | Models on TF Hub | Android (Coming Soon) | Community |
Text Detection | EAST Text Detector (Paper) | Models on TF Hub | Conversion and Inference Notebook | Community |
Image Extrapolation | Models on TF Hub | Colab Notebook | Original Paper | Community |
OCR | Models on TF Hub | Project Repository | Community |
Text
Task | Model | Sample apps | Source |
---|---|---|---|
Question & Answer | DistilBERT | Android | Hugging Face |
Text Generation | GPT-2 / DistilGPT2 | Android | Hugging Face |
Text Classification | Download | Android |iOS | Flutter | tf.org & Community |
Speech
Task | Model | App | Reference | Source |
---|---|---|---|
Speech Recognition | DeepSpeech | Reference | Mozilla |
Speech Synthesis | Tacotron-2, FastSpeech2, MB-Melgan | Android | TensorSpeech |
Speech Synthesis (TTS) | Tacotron2, FastSpeech2, MelGAN, MB-MelGAN, HiFi-GAN, Parallel WaveGAN | Inference Notebook | Project Repository | Community |
Recommendation
Task | Model | App | Reference | Source |
---|---|---|---|
On-device Recommendation | Dual-Encoder | Android | iOS | Reference | tf.org & Community |
There are quite a variety of examples, so it will be useful when developing a smartphone app that is directly utilized or applied with an AI model that has been directly researched and developed. Here is a link to the awesome tensorflow lite site repository: