ARKit Flutter Plugin

Overview

arkit_flutter_plugin

Codemagic build status flutter awesome pub package

Note: ARKit is only supported by mobile devices with A9 or later processors (iPhone 6s/7/SE/8/X, iPad 2017/Pro) on iOS 11 and newer. For some features iOS 12 or newer is required.

Usage

Depend on it

Follow the installation instructions from Dart Packages site.

Update Info.plist

ARKit uses the device camera, so do not forget to provide the NSCameraUsageDescription. You may specify it in Info.plist like that:

    <key>NSCameraUsageDescription</key>
    <string>Describe why your app needs AR here.</string>

Prior to Flutter 1.22, platform views were in developers preview, hence if you want to use the plugin with old Flutter versions you need to add the following code to Info.plist:

    <key>io.flutter.embedded_views_preview</key>
    <string>YES</string>

Write the app

The simplest code example:

import 'package:flutter/material.dart';
import 'package:arkit_plugin/arkit_plugin.dart';
import 'package:vector_math/vector_math_64.dart';

void main() => runApp(MaterialApp(home: MyApp()));

class MyApp extends StatefulWidget {
  @override
  _MyAppState createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> {
  ARKitController arkitController;

  @override
  void dispose() {
    arkitController?.dispose();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) => Scaffold(
      appBar: AppBar(title: const Text('ARKit in Flutter')),
      body: ARKitSceneView(onARKitViewCreated: onARKitViewCreated));

  void onARKitViewCreated(ARKitController arkitController) {
    this.arkitController = arkitController;
    final node = ARKitNode(
        geometry: ARKitSphere(radius: 0.1), position: Vector3(0, 0, -0.5));
    this.arkitController.add(node);
  }
}

Result:

flutter

Examples

I would highly recommend to review the sample from the Example folder. You may find a couple of samples in the Example folder of the plugin. Some samples rely on this Earth image

Name Description Link Demo
Hello World The simplest scene with different geometries. code twitter
Earth Sphere with an image texture and rotation animation. code twitter
Tap Sphere which handles tap event. code twitter
Plane Detection Detects horizontal plane. code twitter
Distance tracking Detects horizontal plane and track distance on it. code twitter
Measure Measures distances code twitter
Physics A sphere and a plane with dynamic and static physics code twitter
Image Detection Detects Earth photo and puts a 3D object near it. code twitter
Network Image Detection Detects Mars photo and puts a 3D object near it. code
Custom Light Hello World scene with a custom light spot. code
Light Estimation Estimates and applies the light around you. code twitter
Custom Object Place custom object on plane. code twitter
Occlusion Spheres which are not visible after horizontal and vertical planes. code twitter
Manipulation Custom objects with pinch and rotation events. code twitter
Face Tracking Face mask sample. code twitter
Panorama 360 photo. code twitter
Custom Animation Custom object animation. Port of https://github.com/eh3rrera/ARKitAnimation code twitter
Widget Projection Flutter widgets in AR code twitter

If you prefer video here is a playlist with "AR in Flutter" videos: AR in Flutter videos

UX advice

You might want to check the device capabilities before establishing an AR session. Review the Check Support sample for the implementation details.

Before you go to AppStore

The plugin supports TrueDepth API. In case you didn't use it, your app will be rejected by Apple. Hence you need to remove any TrueDepth functionality by modifying your Podfile file

post_install do |installer|
  installer.pods_project.targets.each do |target|
    target.build_configurations.each do |config|
      ... # Here are some configurations automatically generated by flutter

      config.build_settings['OTHER_SWIFT_FLAGS'] = '-DDISABLE_TRUEDEPTH_API'
    end
  end
end

FAQ

  • Is it possible to use this plugin on Android?
    No, as ARKit is not available on Android. You might want to try ARCore plugin instead.
  • My app crashes when I open the AR scene several times. Why?
    Most probably that's because you didn't call dispose method on the ARKit controller.
  • One of the features I need is merged in the repository, but is not available on pub.dev. How can I use the latest version?
    You may use the latest version by changing the pubspec.yaml dependency to:
dependencies:
  arkit_plugin:
    git: git://github.com/olexale/arkit_flutter_plugin.git

Contributing

If you find a bug or would like to request a new feature, just open an issue. Your contributions are always welcome!

Comments
  • Use dynamic image in app in ARKitMaterialProperty

    Use dynamic image in app in ARKitMaterialProperty

    Hi,

    I really like your plugin thank you for all your work !

    I would like to know if it's possible to use a dynamic image inside ARKitMaterialProperty (instead of URL or asset image): It's not images I load from the assets, but images I'm building at runtime. I'm doing it using the following code :

     var image = await boundary.toImage(pixelRatio: 1);
        ByteData byteData = await image.toByteData(format: ImageByteFormat.png);
        Uint8List pngBytes = byteData.buffer.asUint8List();
        return pngBytes;
    

    What I'm doing is transforming a rendered widget into a image to display it inside the ARView.

    I've done with ARCore, but I'm struggling to do it with ARKit since it doesn't take ByteData nor Uint8List as argument, is there any workaround ?

    (I'm not projecting them like your example because they need to be placed at very specific places using LatLng coordinates)

    opened by RobinLbt 11
  • Memory deallocation on controller dispose

    Memory deallocation on controller dispose

    In the example app, if I continue to open and close multiple AR screens, the amount of memory used goes up every time I open a screen. It looks like the controller's dispose function pauses the session, but if I open a new page, it appears to initialize a new controller without deallocating the last one.

    I need a similar functionality to the example app where the user can go in and out of the AR screens. Is there a workaround to this?

    opened by miles-au 9
  • [Request] Possibility to add measure feature ?

    [Request] Possibility to add measure feature ?

    Hi!

    I'm new in Flutter and I wanted to integrate ARKit in my project. Thanks to you it's possible!

    I'm very interested by this measure feature, and I would like to know if you can add this feature.

    Thank you in advance!

    enhancement 
    opened by jakoby12 9
  • Panorama Sample not working

    Panorama Sample not working

    just download the project from github and try to run it @olexale in the list when you click on the panorama (360 photo sample) it opens a new window, but the screen is all white.

    opened by maxTeste 8
  • Loading image assets dynamically

    Loading image assets dynamically

    Hello, First, thanks for this awesome plugin !

    I had a question, is it possible to load dynamically image assets in the app ? To be more clear, I want to use the widget projection & Image detection features but I can't load the asset directly in the bundle. Indeed, I want to detect images downloaded directly from the app, so images are not know at advance and can't be added to the project...

    Is there already possible ?

    Thanks in advance.

    opened by istornz 8
  • [Question] How do i track an image (and update the anchor position)

    [Question] How do i track an image (and update the anchor position)

    Hello,

    i would like to track an image. For that i have to get the new anchor positions (or the position of the image) every X secounds. How do i do that?

    Best Regards

    opened by vuk64 7
  • App rejection by TrueDepth API

    App rejection by TrueDepth API

    Can I choose to use the TrueDepth API?

    I'm sorry to hear that you recently received a decline in the review process for your app using this plugin.

    The app was rejected because of the code in the TrueDepth API included in arkit_plugin. Since the app uses only the AR View feature, the TrueDepth API is not required, but it appears to be rejected because it is included in the plugin.

    Can't we update the TrueDepth API so that we can pick up the answer from the Apple Review Team?


    Guideline 2.5.1-Performance-Software Requirements

    During review, we found that your app includes TrueDepth APIs. However, we were not able to locate any features in your app that use TrueDepth APIs.

    Next steps

    If your app does not include any features that use TrueDepth APIs, please remove them from your app.

    If your app uses the Unity ARKit plugin, it would be appropriate to update to the latest version (https://forum.unity.com/threads/submitting-arkit-apps-to-appstore-without-face-tracking.504572/ # post-3297235), which includes a setting that allows you to exclude TrueDepth APIs.

    If your app does include features that use TrueDepth APIs, please reply to this message in Resolution Center to provide information on how to locate them.

    Please see attached screenshots for details.

    opened by Nickkun 7
  • Possible to load .dae (or any extension) from network to serve as 3d model/asset?

    Possible to load .dae (or any extension) from network to serve as 3d model/asset?

    Duplicate of https://github.com/olexale/arkit_flutter_plugin/issues/62 basically, but can't seem to get this to work. I've been playing with ARKit (this plugin) and ARCore (https://github.com/giandifra/arcore_flutter_plugin). With ARCore, to load a 3d model, it was a matter of adding an objectUrl argument to a Node, in this case a .gltf file hosted from GitHub (raw).

    Is there something similar with this plugin? From my understanding, the ARKitReferenceNode 'url' property simply means a local model. That works, but again that's local. Can't find anything in the example that's loading a 3d model from a remote URL.

    Also tried Googling around, can't find clear answers but it seems like other developers are downloading 3d models beforehand (e.g. during a loading screen) and then load them like a local model.

    I've tried to do the same, by simply putting a 3d model in the flutter root 'assets' folder but that doesn't work either, it crashes.

    This merge https://github.com/olexale/arkit_flutter_plugin/pull/61 sounds like the 'url' property should now load assets outside the main bundle, but when I tried using 'assets/nameofmodel.dae' (and '/assets/nameofmodel.dae') it crashed the app instantly (right after its trying to load model from URL path I believe) without an error message, even in verbose mode. It's just ending the logs with lost connection to device and shutting down messages.

    Seems like the only way to get a 3d model is to include it in the main bundle so far, really would like to use models downloaded during runtime.

    opened by BrutalCoding 6
  • Record camera question

    Record camera question

    Is there a way to record video? I'm currently using the [camera plugin](https://pub.dev/packages/camera) to record video but I want to be able to use arkit as well. Is this possible?

    opened by gusabdala 6
  • Can I make a double sided sphere?

    Can I make a double sided sphere?

    Hi, thanks for the awesome plugin. I wonder if can make a double sided sphere, to simulate a 360 view of a photo. Just like this example: https://stackoverflow.com/questions/46423415/how-to-apply-360-video-as-a-texture-in-arkit

    Or if you happen to know how I can make this somehow I would be infinitely grateful.

    Thank you.

    opened by gustavofurtado 6
  • ARKitReferenceNode Material. Add Request.

    ARKitReferenceNode Material. Add Request.

    Hello @olexale.

    I see we can change Material of Basic Geometries Like Sphere,Cube etc.

    But in real world Problems where we want to Change Material Properties of custom Objects.

    Do you have plan for adding Material diffusion and rotation for ARKitReferenceNode.

    Thanks !!

    opened by sikandernoori 5
  • ARFaceGeometry

    ARFaceGeometry

    Hi, I am looking to calculate the width of a face from precise points, is ARFaceGeometry available or any other method that can allow this calculation? Thanks in advance.

    opened by Mariollet 0
  • Set depth

    Set depth

    Hi, we are developing a project to help in home devices installation. We are doing good detection the of the plane, but after that we are having problems with depth. We want for example to set the corner of a room to put a devices in that corner 2mts high.

    opened by leandrossi 0
  • Body Tracking not working

    Body Tracking not working

    I just git cloned the repo and executed on my iPhone X, iOS 15.5 trought xcode/execute/run on device.

    All the others examples working good, but body tracking just show a black screen.

    Any suggestion ?

    opened by marcojr 0
Releases(v1.0.5)
Owner
Oleksandr Leuschenko
Oleksandr Leuschenko
Flutter Plugin for AR (Augmented Reality) - Supports ARKit on iOS and ARCore on Android devices

ar_flutter_plugin Flutter Plugin for AR (Augmented Reality) - Supports ARKit for iOS and ARCore for Android devices. Many thanks to Oleksandr Leuschen

Lars Carius 222 Jan 4, 2023
This is just the simplyfied Flutter Plugin use for one of the popular flutter plugin for social media login.

social_media_logins Flutter Plugin to login via Social Media Accounts. Available Social Media Logins: Facebook Google Apple Getting Started To use thi

Reymark Esponilla 3 Aug 24, 2022
Permission plugin for Flutter. This plugin provides a cross-platform (iOS, Android) API to request and check permissions.

On most operating systems, permissions aren't just granted to apps at install time. Rather, developers have to ask the user for permissions while the

Baseflow 1.7k Jan 3, 2023
A Flutter example to use Google Maps in iOS and Android apps via the embedded Google Maps plugin Google Maps Plugin

maps_demo A Flutter example to use Google Maps in iOS and Android apps via the embedded Google Maps plugin Google Maps Plugin Getting Started Get an A

Gerry High 41 Feb 14, 2022
Permission plugin for Flutter. This plugin provides a cross-platform (iOS, Android) API to request and check permissions.

Flutter permission_handler plugin The Flutter permission_handler plugin is build following the federated plugin architecture. A detailed explanation o

Baseflow 1.7k Dec 31, 2022
Unloc customizations of the Permission plugin for Flutter. This plugin provides an API to request and check permissions.

Flutter Permission handler Plugin A permissions plugin for Flutter. This plugin provides a cross-platform (iOS, Android) API to request and check perm

Unloc 1 Nov 26, 2020
Klutter plugin makes it possible to write a Flutter plugin for both Android and iOS using Kotlin only.

The Klutter Framework makes it possible to write a Flutter plugin for both Android and iOS using Kotlin Multiplatform. Instead of writing platform spe

Gillian 33 Dec 18, 2022
A Flutter step_tracker plugin is collect information from user and display progress through a sequence of steps. this plugin also have privilege for fully customization from user side. like flipkart, amazon, myntra, meesho.

step_tracker plugin A Flutter step_tracker plugin is collect information from user and display progress through a sequence of steps. this plugin also

Roshan nahak 5 Oct 21, 2022
Boris Gautier 1 Jan 31, 2022
File picker plugin for Flutter, compatible with both iOS & Android and desktop (go-flutter).

File Picker A package that allows you to use the native file explorer to pick single or multiple files, with extensions filtering support. Currently s

Miguel Ruivo 985 Jan 5, 2023
Plugin to access VPN service for Flutter | Flutter 的 VPN 插件

Flutter VPN plugin This plugin help developers to access VPN service in their flutter app. 本插件帮助开发者在自己的应用内调用 VPN 服务。 The Android part was implemented

Xdea 277 Dec 28, 2022
Flutter Music Player - First Open Source Flutter based material design music player with audio plugin to play local music files.

Flutter Music Player First Open Source Flutter based Beautiful Material Design Music Player(Online Radio will be added soon.) Demo App Play Store BETA

Pawan Kumar 1.5k Jan 8, 2023
Flutter Music Player - First Open Source Flutter based material design music player with audio plugin to play local music files.

Flutter Music Player First Open Source Flutter based Beautiful Material Design Music Player(Online Radio will be added soon.) Demo App Play Store BETA

Pawan Kumar 1.5k Jan 2, 2023
Google places picker plugin for flutter. Opens up the google places picker on ios and android returning the chosen place back to the flutter app.

flutter_places_dialog Shows a places picker dialog in ios and android, returning the data in the places picker to the app. Getting Started Generate yo

null 44 Dec 6, 2022
A new flutter plugin for mapbox. You can use it for your map backgrounds inside flutter applications

A new flutter plugin for mapbox. You can use it for your map backgrounds inside flutter applications

Boris Gautier 5 Sep 14, 2022
Flutter plugin for Flutter desktop(macOS/Linux/Windows) to change window size.

desktop_window Flutter plugin for Flutter desktop(macOS/Linux/Windows) to change window size. Usage import 'package:desktop_window/desktop_window.dart

ChunKoo Park 72 Dec 2, 2022
A flutter plugin for Easily make Flutter apps responsive. Automatically adapt UI to different screen sizes. Responsiveness made simple.

A flutter plugin for Easily make Flutter apps responsive. Automatically adapt UI to different screen sizes. Responsiveness made simple.

Urmish Patel 191 Dec 29, 2022
Flutter plugin for Flutter desktop(macOS/Linux/Windows) to change window size.

desktop_window Flutter plugin for Flutter desktop(macOS/Linux/Windows) to change window size. Usage import 'package:desktop_window/desktop_window.dart

ChunKoo Park 72 Dec 2, 2022
A flutter plugin for integrating Mobile Money Payments to your flutter apps

Add Mobile Money payments to your flutter apps using the FlutterWave api gateway. Features Recieve Payments through Mobile Money in Uganda Supports MT

null 5 Nov 9, 2022