Flutter realtime object detection with Tensorflow Lite

Overview

Flutter realtime object detection with Tensorflow Lite

Flutter realtime object detection with Tensorflow Lite

Info

An app made with Flutter and TensorFlow Lite for realtime object detection using model YOLO, SSD, MobileNet, PoseNet.

⭐ Features

  • Realtime object detection on the live camera

  • Using Model: YOLOv2-Tiny, SSDMobileNet, MobileNet, PoseNet

  • Save image has been detected

  • MVVM architecture


πŸš€   Installation

  1. Install Packages
camera: get the streaming image buffers
https://pub.dev/packages/camera
tflite: run model TensorFlow Lite
https://pub.dev/packages/tflite
provider: state management
https://pub.dev/packages/provider

2. Configure Project
  • Android
android/app/build.gradle

android {
    ...
    aaptOptions {
        noCompress 'tflite'
        noCompress 'lite'
    }
    ...
}


minSdkVersion 21

3. Load model
loadModel() async {
    Tflite.close();
    await Tflite.loadModel(
        model: "assets/models/yolov2_tiny.tflite",  
        //ssd_mobilenet.tflite, mobilenet_v1.tflite, posenet_mv1_checkpoints.tflite
        labels: "assets/models/yolov2_tiny.txt",    
        //ssd_mobilenet.txt, mobilenet_v1.txt
        //numThreads: 1, // defaults to 1
        //isAsset: true, // defaults: true, set to false to load resources outside assets
        //useGpuDelegate: false // defaults: false, use GPU delegate
    );
  }

4. Run model

For Realtime Camera

  //YOLOv2-Tiny
  Future<List<dynamic>?> runModelOnFrame(CameraImage image) async {
     var recognitions = await Tflite.detectObjectOnFrame(
          bytesList: image.planes.map((plane) {
            return plane.bytes;
          }).toList(),
          model: "YOLO",
          imageHeight: image.height,
          imageWidth: image.width,
          imageMean: 0,                 // defaults to 127.5
          imageStd: 255.0,              // defaults to 127.5
          threshold: 0.2,               // defaults to 0.1
          numResultsPerClass: 1,
        );   
    return recognitions;
  }

  //SSDMobileNet
  Future<List<dynamic>?> runModelOnFrame(CameraImage image) async {
     var recognitions = await Tflite.detectObjectOnFrame(
          bytesList: image.planes.map((plane) {
            return plane.bytes;
          }).toList(),
          model: "SSDMobileNet",
          imageHeight: image.height,
          imageWidth: image.width,
          imageMean: 127.5,
          imageStd: 127.5,
          threshold: 0.4,
          numResultsPerClass: 1,
        );   
    return recognitions;
  }

  //MobileNet
  Future<List<dynamic>?> runModelOnFrame(CameraImage image) async {
     var recognitions = await Tflite.runModelOnFrame(
          bytesList: image.planes.map((plane) {
            return plane.bytes;
          }).toList(),
          imageHeight: image.height,
          imageWidth: image.width,
          numResults: 5
        );   
    return recognitions;
  }

  //PoseNet
  Future<List<dynamic>?> runModelOnFrame(CameraImage image) async {
     var recognitions = await Tflite.runPoseNetOnFrame(
          bytesList: image.planes.map((plane) {
            return plane.bytes;
          }).toList(),
          imageHeight: image.height,
          imageWidth: image.width,
          numResults: 5
        );   
    return recognitions;
  }

For Image

  Future<List<dynamic>?> runModelOnImage(File image) async {
    var recognitions = await Tflite.detectObjectOnImage(
        path: image.path,
        model: "YOLO",
        threshold: 0.3,
        imageMean: 0.0,
        imageStd: 127.5,
        numResultsPerClass: 1
    );
    return recognitions;
  }
Output format:

YOLO,SSDMobileNet
  [{
    detectedClass: "dog",
    confidenceInClass: 0.989,
    rect: {
        x: 0.0,
        y: 0.0,
        w: 100.0,
        h: 100.0
    }
  },...]

MobileNet
[{
    index: 0,
    label: "WithMask",
    confidence: 0.989
  },...]

PoseNet
[{
    score: 0.5,
    keypoints: {
        0: {
            x: 0.2,
            y: 0.12,
            part: nose,
            score: 0.803
        },
        1: {
            x: 0.2,
            y: 0.1,
            part: leftEye,
            score: 0.8666
        },
        ...
    }
  },...]


5. Issue
* IOS
Downgrading TensorFlowLiteC to 2.2.0

Downgrade your TensorFlowLiteC in /ios/Podfile.lock to 2.2.0
run pod install in your /ios folder

6. Source code
please checkout repo github
https://github.com/hiennguyen92/flutter_realtime_object_detection

πŸ’‘ Demo

  1. Demo Illustration: https://www.youtube.com/watch?v=__i7PRmz5kY&ab_channel=HienNguyen
  2. Image
You might also like...

Flutter shareable package of object-oriented classes for local caching of user data in json

Flutter shareable package of object-oriented classes for local caching of user data in json

json_cache Json Cache is an object-oriented package to serve as a layer on top of local storage packages - packages that persist data locally on the u

Dec 19, 2022

Object-oriented package for validating Flutter form fields.

Object-oriented package for validating Flutter form fields.

formdator Contents Overview Getting Started List of Validators Categories Demo Application References Overview Form Validator β€” Formdator is a fully o

Oct 26, 2022

A Flutter 3D widget that renders Wavefront's object files.

A Flutter 3D widget that renders Wavefront's object files.

Flutter Cube A Flutter 3D widget that renders Wavefront's object files. Getting Started Add flutter_cube as a dependency in your pubspec.yaml file. de

Dec 22, 2022

This is a flutter plugin to detect edges in a live camera, take the picture of detected edges object, crop it, and save.

This is a flutter plugin to detect edges in a live camera, take the picture of detected edges object, crop it, and save.

edge_detection A flutter plugin to detect edges of objects, scan paper, detect corners, detect rectangles. It allows cropping of the detected object i

Dec 28, 2022

Receiving ozh's github-colors repository with latest commit of colors.json to Flutter's Color object.

Receiving ozh's github-colors repository with latest commit of colors.json to Flutter's Color object.

Apply GitHub's languages colours into Flutter's Color object. Receiving ozh's github-colors repository with latest commit of colors.json to Flutter's

Jun 6, 2022

Return a result ErrorOr with either a value T or an error Object.

ErrorOr Return a result ErrorOr with either a value T or an error Object. Features Always return a value ErrorOr from an async function. Let the calle

Nov 6, 2022

🎯 This library automatically generates object classes from JSON files that can be parsed by the freezed library.

🎯 This library automatically generates object classes from JSON files that can be parsed by the freezed library.

The Most Powerful Way to Automatically Generate Model Objects from JSON Files ⚑ 1. Guide 🌎 1.1. Features πŸ’Ž 1.1.1. From 1.1.2. To 1.2. Getting Starte

Nov 9, 2022

Face detection app built with Flutter and Firebase ML Kit

Face detection app built with Flutter and Firebase ML Kit

Flutter Face Detection Flutter Face Detection with Firebase ML Kit and CustomPainter. Read the article on Medium Made with 😍 by Akora-IngDKB. Follow

Sep 29, 2022

Detectable text field - Flutter Text widgets with detection features

Detectable text field - Flutter Text widgets with detection features

detectable_text_field Text widgets with detection features. You can detect hasht

Feb 2, 2022
Owner
null
Face Mask Detection mobile application built with Flutter and TensorFlow lite in order to detect face masks using images and live camera.

Face Mask Detector App Face Mask Detection mobile application built with Flutter and TensorFlow lite in order to detect face masks using images and li

Yousef Shaban 3 Aug 15, 2022
Simple face recognition authentication (Sign up + Sign in) written in Flutter using Tensorflow Lite and Firebase ML vision library.

FaceNetAuthentication Simple face recognition authentication (Sign up + Sign in) written in Flutter using Tensorflow Lite and Google ML Kit library. S

Marcos Carlomagno 279 Jan 9, 2023
A Flutter plugin to access TensorFlow Lite apis.

tensorflow_lite A Flutter plugin to access TensorFlow Lite apis. TensorFlow Lite is TensorFlow’s lightweight solution for mobile and embedded devices.

Kashif Minhaj 74 Nov 22, 2022
TensorFlow Lite Flutter Plugin

Overview TensorFlow Lite Flutter plugin provides a flexible and fast solution for accessing TensorFlow Lite interpreter and performing inference. The

Amish Garg 383 Jan 5, 2023
Real-time object detection in Flutter using camera and tflite plugin

For details: https://medium.com/@shaqian629/real-time-object-detection-in-flutter-b31c7ff9ef96 flutter_realtime_detection Real-time object detection i

Post_Swift 6 Oct 12, 2022
Flutterbodydetection - A flutter plugin that uses MLKit on iOS/Android platforms to enable body pose and mask detection using Pose Detection and Selfie Segmentation APIs for both static images and live camera stream.

body_detection A flutter plugin that uses MLKit on iOS/Android platforms to enable body pose and mask detection using Pose Detection and Selfie Segmen

null 18 Dec 5, 2022
Learn how to build a tensorflow model on Techable Machine and then run it on flutter app.

Ml With Flutter Learn how to build a tensorflow model on Techable Machine and then run it on flutter app. Youtube Tutorial Show Support Recommend Me O

Sanskar Tiwari 133 Jan 3, 2023
Lite version of smart_select package, zero dependencies, an easy way to provide a single or multiple choice chips.

Lite version of smart_select package, zero dependencies, an easy way to provide a single or multiple choice chips. What's New in Version 2.x.x Added p

Irfan Vigma Taufik 97 Dec 15, 2022
Lite-graphql - A light way implementation of GraphQL client in dart language

lite GraphQL client A light way implementation of GraphQL client in dart languag

Vincenzo Palazzo 3 Mar 17, 2022
Interactive command line interface Couchbase Lite REPL utility built with the Dart

Couchbase Lite Dart CLI Interactive command line interface Couchbase Lite REPL utility built with the Dart programming language. This code uses the cb

Pieter Greyling 2 Jul 20, 2022