Drishti is an open-source cross-platform mobile application project at Incubate Nepal that incorporates Machine Learning and Artificial Intelligence

Overview

Drishti Logo

Build Status App Version License GitHub Stars

Drishti (Nepali: दृष्टि | English: Vision) is an open-source cross-platform mobile application project at Incubate Nepal that incorporates Machine Learning and Artificial Intelligence to help visually impaired people recognize different currency bills and perform daily cash transactions more effectively.

We plan to expand Drishti to other applications like Short Text and Document Reading in the future.


Table of Contents


General Information

The mobile application is used by the visually impaired community to recognize different currency bills and perform their daily cash transactions more easily. This user-friendly app uses the phone camera, accessibility features, and gestures to provide the desired results.


Project Deliverables

  • Mobile Application
  • Computer Vision Model
  • Website

Installation

The app is now available on Google Play Store as Drishti Nepal.

Drishti Google Play Link


App Demo

demo.mp4

Screenshots

Drishti App Screenshots


Technologies

  • Flutter
  • Python
  • Tensorflow

Techstack

  • Frontend: Flutter
  • Data science:
    • Experimentation: PyTorch
    • Training: Tensorflow
    • Deployment: Tflite (Tensorflow Lite)
  • Website: HTML, CSS, JavaScript

Features

List of implemented features:

  • Audio feedback in Native Language (Nepali)
  • Ad-free
  • Available to use Offline
  • History Feature
    • Keep Track of Scanned Notes
    • Sum up the transactions
    • Classify the Scanned Notes ("Today", "This Week", "This Month")

TODOs for future development:

  • Add haptic feedback for Cash Recognition
  • Implement Short Text and Document Reading (Nepali)

Status

  • Cash Recognition Feature
  • Short Text and Document Reading

Inspiration

Drishti app is particularly developed for the blind and low vision community of Nepal. To understand the need of our end users, the team studied and empathized with the blind and low-vision individuals from Blind Youth Association Nepal rather than digging into the financial problems faced by them.

After interviewing more than 50 visually impaired individuals, we found that they are already using some cash recognizing apps but are not satisfied with them. Some interviewees also shed light on their daily difficulties while traveling. They have suggested to include some additional features of location tracking and user friendly GPS in our future iterations. Our basic goal is to make a robust application which could benefit them.

As a/an I want to So that I can Status of progress
User Scan the currency notes at a bank Confirm the total sum of money I received Done
User Identify the bills in the dark using my flashlight Transact money during the evening too Done
Developer Get the feedback from the users Make necessary upgrades in the application Done
Product Owner Know the progress made in the application during each iteration Prioritize the future goals for the project Done
User Recognize the bills without having to ask others Be independent in my financial affairs Done
User Have a simple UI in my application Easily navigate into the app Done
User Have the buttons labeled in the app Properly recognize the buttons using my phone’s screen reader Done
Software engineer Easily get the source code of the app Modify the app and improve its accuracy Done
User See the history of my scanned notes in a day I can keep a record of my expenditure list conveniently Done

Contributing

These are the ways in which you can contribute to this project:

  • Submit bugs and feature requests
  • Review source code changes
  • Contribute directly to improve Drishti (adding new features and fixing bugs). The steps to do so include:
    • Fork this repository
    • Clone the forked repository to your machine
    • Create a branch with your GitHub username
    • Commit your changes, push the changes to GitHub
    • Open a pull request here

Contact Us


License

Drishti is MIT licensed, as found in the LICENSE file.

Comments
  • Null Safety Migration

    Null Safety Migration

    Changes Made:

    1. Migrated Code to Null Safety
    2. Changed kotlin version to 1.4.32 in android/build.gradle
    3. Updated all dependencies to latest version

    Tested Flutter Version : 2.2.3 Channel Stable

    opened by aadarshadhakalg 3
  • App does not open when Android debug option is on

    App does not open when Android debug option is on

    First of, this is a great initiative. Good to know about the project.

    The issue is, I just installed the app and while opening the app, the messaged popped up and it did not allow me to open the app.

    I am just curious to know about it. Why are we not allowing the app to run in such environment? I believe it would help to debug better.

    Android Version: 6.0 Phone: Huawei

    Thanks.

    opened by anitsh 2
  • Kotlin Build Error

    Kotlin Build Error

    Running an app produces error: Execution failed for task ':audioplayers:compileDebugKotlin'.

    I could run app by upgrading kotlin version. But is there any proper way to handle this?

    @basnetsoyuj @Pranjal33 @Oshin-hi

    opened by iamsannyrai 1
  • iamsannyrai

    iamsannyrai

    1. Removes code duplication in main_screen.dart
    2. Change from onTap to onDoubleTap in according to tooltip description.
    3. Adds two different vibration patterns when cash recognition is either successful or failed.
    opened by iamsannyrai 0
  • ML code to train model

    ML code to train model

    I was looking at the code and was wondering about the way to train the model. Also, I don't see any datasets that are being used to train and recognize the cash. Are you guys running the script in any cloud platform to train the model? Can I get to see the code to train the model?

    opened by techylax 0
  • Implement a Real-time Recognizing Feature

    Implement a Real-time Recognizing Feature

    Currently, our app gives the audio output after double-tapping the screen. The app should be able to recognize the notes as soon as it is placed and give audio results. Double-tapping the screen to hear the output is not feasible for our users.

    enhancement help wanted 
    opened by pranjalchalise 3
  • Add Flashlight Feature.

    Add Flashlight Feature.

    Recognizing notes in dim light and nights is a huge functional requirement for our users. Due to time constraints, the team could not implement the flashlight feature in the camera package.

    enhancement help wanted 
    opened by pranjalchalise 6
  • Model Accuracy and App Size

    Model Accuracy and App Size

    Work on improving model accuracy, simultaneously reducing the app size. The app was not supported by many devices due to its large application size. Since it is not 100% accurate, users are skeptical of the app. The developers have a task to improve the accuracy while reducing the app size at the same time.

    enhancement help wanted 
    opened by pranjalchalise 11
  • Improve SDK Compatibility.

    Improve SDK Compatibility.

    After user testing, we found out that our app does not work on older versions of Android. To ensure that our app reaches many of the visually impaired users, the application has to improve its SDK compatibility. Current Compatibility- minSdkVersion 23 targetSdkVersion 29

    enhancement help wanted 
    opened by pranjalchalise 0
  • Add haptic feedback for cash recognition.

    Add haptic feedback for cash recognition.

    Add different haptic feedback (vibrations) for different notes (might be helpful in places where the user can't listen to the audio).

    This issue has partially been fixed in this commit: https://github.com/DrishtiNP/Drishti/commit/c7c8750b0bd0b065a5a831ff468835b22afc4744 by @iamsannyrai.

    (Gives different haptic feedback when a note is detected vs note isn't detected)

    We still need to integrate different vibrations of different patterns/intensities for each individual note.

    enhancement help wanted 
    opened by pranjalchalise 1
Owner
Drishti Nepal
Drishti Nepal
Simple and modern news app that incorporates REST API (newsapi.org), all built entirely with Flutter.

A simple news app with a minimalistic and clean UI that incorporates the newsapi.org api all built entirely with Flutter. Be sure to leave a star ??

Carlton Aikins 73 Dec 1, 2022
Machine learning algorithms in Dart programming language

Machine learning algorithms for Dart developers What is the ml_algo for? The main purpose of the library is to give native Dart implementation of mach

Ilya Gyrdymov 134 Jan 4, 2023
Proyecto de aplicación móvil con inteligencia artificial

INTERFACES | | | | | | visionIA-app Proyecto de aplicación móvil con inteligencia artificial Planteamiento del proyecto. Este proyecto se basa en el d

Saul Esteban Urrea Osorio 3 Nov 17, 2021
A beautiful and cross platform NHentai Client, Support desktop and mobile phone

A beautiful and cross platform NHentai Client. Support desktop and mobile phone (Mac/Windows/Linux/Android/IOS).

null 324 Dec 29, 2022
KeepReading - a Mobile (Android/IOS) Cross-Platform App wrapper for the Keep Talking and Nobody Explodes manual

KeepReading is a Mobile (Android/IOS) Cross-Platform App wrapper for the Keep Talking and Nobody Explodes manual. It helps you to jump directly to a specific section of the manual with just a tap ?? !

Nicolò Vescera 2 May 16, 2022
Quiz App is cross-platform mobile app, that allows you to test your knowledge on various technologies through quizzes. It's built with Flutter & Dart

Quiz App is cross-platform mobile app, that allows you to test your knowledge on various technologies through quizzes. It's built with Flutter & Dart

Régis 6 Sep 19, 2022
Timy - open source mobile app for groups to communicate and organize themselves. Built with flutter.

Timy app An amazing open-source group messaging app build with flutter. ✨ Main Features Multiple groups (similar to Teams in Slack). Multiple open or

null 1.9k Dec 25, 2022
freeCodeCamp's open-source mobile app

freeCodeCamp.org's open-source mobile app freeCodeCamp.org is a friendly community where you can learn to code for free. Our full-stack web developmen

freeCodeCamp.org 248 Jan 9, 2023
WalletConnect is an open source protocol for connecting decentralised applications to mobile wallets with QR code scanning or deep linking.

WalletConnect is an open source protocol for connecting decentralised applications to mobile wallets with QR code scanning or deep linking. A user can

Tomas Verhelst 87 Dec 28, 2022
Cross platform application for iOS and Android using Google's SDK Flutter.

scout Cross platform application for iOS and Android using Google's SDK Flutter. Launch screen for the application. The menu for selecting cookies. Cu

null 0 Nov 9, 2021
A cross platform application written in flutter to help people stick to their routines and achieve their goals

Scheduler Scheduler is a cross platform application written in flutter to help people stick to their routines and achieve their goals. Our service inc

Sidheshwar S 3 Jan 21, 2022
Relive is a cross-platform application that can be used both in Android/IOS and it is made using Flutter

?? Introduction Relive is a cross-platform application that can be used both in Android/IOS and it is made using Flutter. It aims to solve the mental

Akshat Tripathi 14 Nov 4, 2022
Cross-platform Twitch Chat application with 3rd-party addon support!

Chatsen Chatsen is a cross-platform application that allows you to chat on Twitch with support for 3rd-party services such as 7TV, BTTV and FFZ. It al

Chatsen 172 Dec 31, 2022
Flutter UI challenge (with Box2D physic)- Smart washing machine app

Flutter UI Challenge- SMART Washing Machine About project The application was written based on this great UI concept: https://dribbble.com/shots/11018

Tomasz Pawlikowski 754 Dec 27, 2022
Aplikasi Android konsultasi dokter dan vending machine

doctor_apps A new Flutter project. Getting Started This project is a starting point for a Flutter application. A few resources to get you started if t

Heri Nur Alim 1 Nov 11, 2021
A library for finite state machine realization

A library for finite state machine realization in Dart. Inspired by Tinder StateMachine library. Usage A simple usage example (using dfunc library for

Kirill Bubochkin 6 Dec 6, 2021
Fluffyboard is a open source, free and self-hostable Whiteboard Application

Fluffyboard Fluffyboard is a open source, free and self-hostable Whiteboard Application Test the demonstration of Fluffyboard Table of Contents Featur

Yonggan 59 Dec 13, 2022