Extensible Dart interpreter for Dart with full interop

Related tags

Utilities dart_eval
Overview

Pub Version

dart_eval is an extensible interpreter for the Dart language, written in Dart. It's powered under the hood by the Dart analyzer, so it achieves 100% correct and up-to-date parsing (although evaluation isn't quite there yet.)

The primary goal of dart_eval is to be interoperable with real Dart code. Classes created in 'real Dart' can be used inside the interpreter with a wrapper, and classes created in the interpreter can be used outside it by creating an interface and bridge class.

For now, the project's current target is to achieve 100% correct evaluation of valid Dart code. Correct error handling (beyond parse errors) is out of the scope at this time.

Usage

A simple usage example:

import 'package:dart_eval/dart_eval.dart';

main() {
  final parser = Parse();

  final scope = parser.parse('''
      class Cat {
        Cat();
        void speak(String name) {
          print('meow');
          print(name);
        }
      }
      void main() {
        final cat = Cat();
        cat.speak('Fluffy');
      }
  ''');

  scope('main', []);
}

Interop

There are three types of interop:

  • Value interop
  • Wrapper interop
  • Bridge interop

Value interop

Value interop is the most basic form, and happens automatically whenever the Eval environment is working with an object backed by a real Dart value. (Therefore, an int and a string are value interop enabled, but a class created inside Eval isn't.) To access the backing object of an EvalValue, use its realValue property. You can also pass a value-interop only enabled object to Eval using EvalRealObject with its optional parameters not set, but this is not recommended. Instead, you should use the class pertaining to the value type, such as EvalInt or EvalString.

Wrapper interop

Using a wrapper enables the Eval environment to access the functions and fields on a class created outside Eval. It's much more powerful than value interop, but much simpler than bridge interop, making it a great choice for certain use cases. To use wrapper interop, create an EvalRealObject using its optional parameters to map out the fields and methods of the wrapped type.

Bridge interop

Bridge interop enables the most functionality: Not only can Eval access the fields of an object, but it can also be extended, allowing you to create subclasses within Eval and use them outside of Eval. For example, bridge interop is used by Flightstream to enable the creation of custom Flutter widgets within Eval. The downside of bridge interop is that it's comparatively difficult to use, and it can't be used to wrap existing objects created in code you don't control. (For utmost flexibility at the expense of simplicity, you can use both bridge and wrapper interop.)

Since Bridge interop requires a lot of boilerplate code, in the future I will be creating a solution for code-generation of that boilerplate.

An example featuring bridge interop is available in the example directory.

Contributing

See Contributing.

FAQ

How does it work?

dart_eval is a fully Dart-based implementation of an interpreter. First, we use the Dart analyzer to parse the code into an AST (abstract syntax tree). Then, we map this to our own AST which is comprised of classes that 'understand' how to evaluate themselves.

Evaluation has two main steps: first, we 'declare' everything, assigning the scope of every part of the code (basically, grouping all of the declarations into fancy Maps so they can be quickly accessed via a lookup, and then giving them references to those Maps so they too can lookup other classes). Then, we simply take a top-level node and execute it, which then calls all of the child nodes under it.

Does it support Flutter?

Yes! Well, kind of. Support for Flutter is not built in but can be added via Bridge interop. Testing has been done to a limited extent and it works. In the future this project will expand support for Flutter.

How fast is it?

Preliminary testing shows that, for simple code, dart_eval running in AOT-compiled Dart is around 11x slower than standard AOT Dart. For many use cases this actually doesn't matter too much, e.g. in the case of Flutter where the app spends 99% of its performance budget in the Flutter framework itself. In the future you can expect this project to have somewhat better performance, although it will always be notably slower than native Dart code.

Potential design issues

This project is somewhat of an abuse of the Dart compiler as, despite best efforts, a lot of code resorts to the use of the dynamic type. This could potentially cause long compile times and increased code size in large apps. Preliminary testing shows it's not an issue for smaller apps.

Features and bugs

Please file feature requests and bugs at the issue tracker.

Comments
  • An example for implementing Flutter with this lib

    An example for implementing Flutter with this lib

    Hi @ethanblake4, this is truly a great lib & exactly what I was looking for. You have mentioned that we can use Flutter widgets with this lib. I tried to add them using the example you have mentioned. But from my approach, it seems I have to add boilerplate code for all the required Widgets ie for text, container, etc. Is there a better way to approach this? Could you share an example, if possible? Thanks.

    opened by SahajRana 13
  • passing an instance of a domain model into Eval ?

    passing an instance of a domain model into Eval ?

    I asked something similar in the flutter_eval project, but now that I think about it, this is probably more of a dart_eval thing.

    I have a package with a bunch of class definitions that contain the main models for my project.

    I would like to pass an instance of one of these models into the eval'd code and use some of the model properties in the eval'd code.

    Is there a way to do something like that?

    opened by aktxyz 12
  • Support String.substring()

    Support String.substring()

    I'm trying to figure out what is supported for core types. Eg I have a simple test harness:

    import 'package:dart_eval/dart_eval.dart';
    
    void main(List<String> arguments) {
      final program = '''
          int main() {
              // my test code goes here
          }
      ''';
    
      print("eval result:${eval(program)}");
    }
    

    If my program text is:

    bool main() {
            final cat = 'Fluffy';
            return cat.isEmpty;
          }
    

    I get the expected result: eval result:false

    For:

    String main() {
            final cat = 'Fluffy';
            return cat.substring(0,2);
          }
    

    However I get:

    Unhandled exception:
    Null check operator used on a null value
    #0      resolveInstanceMethod (package:dart_eval/src/eval/compiler/expression/method_invocation.dart:201:62)
    #1      new AlwaysReturnType.fromInstanceMethod (package:dart_eval/src/eval/compiler/type.dart:438:16)
    #2      AlwaysReturnType.fromInstanceMethodOrBuiltin (package:dart_eval/src/eval/compiler/type.dart:471:28)
    #3      Variable.invoke (package:dart_eval/src/eval/compiler/variable.dart:210:28)
    #4      _invokeWithTarget (package:dart_eval/src/eval/compiler/expression/method_invocation.dart:135:14)
    #5      compileMethodInvocation (package:dart_eval/src/eval/compiler/expression/method_invocation.dart:29:12)
    #6      compileExpression (package:dart_eval/src/eval/compiler/expression/expression.dart:27:12)
    #7      compileReturn (package:dart_eval/src/eval/compiler/statement/return.dart:38:17)
    #8      compileStatement (package:dart_eval/src/eval/compiler/statement/statement.dart:24:12)
    #9      compileBlock (package:dart_eval/src/eval/compiler/statement/block.dart:16:20)
    #10     compileFunctionDeclaration (package:dart_eval/src/eval/compiler/declaration/function.dart:49:14)
    #11     compileDeclaration (package:dart_eval/src/eval/compiler/declaration/declaration.dart:18:5)
    #12     Compiler.compileSources.<anonymous closure>.<anonymous closure> (package:dart_eval/src/eval/compiler/compiler.dart:248:9)
    #13     _LinkedHashMapMixin.forEach (dart:collection-patch/compact_hash.dart:614:13)
    #14     Compiler.compileSources.<anonymous closure> (package:dart_eval/src/eval/compiler/compiler.dart:237:13)
    #15     _LinkedHashMapMixin.forEach (dart:collection-patch/compact_hash.dart:614:13)
    #16     Compiler.compileSources (package:dart_eval/src/eval/compiler/compiler.dart:234:30)
    #17     Compiler.compile (package:dart_eval/src/eval/compiler/compiler.dart:80:12)
    #18     eval (package:dart_eval/src/eval/eval.dart:29:28)
    #19     main (file:///home/maks/work/dart_eval/example/minimal_example.dart:11:24)
    #20     _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:295:32)
    #21     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:192:12)
    

    Is this because only Getters/Setters are curently supported? The reason I'm trying this is to see if its possible now or in future for dart_eval to do "transparent" interop with "real dart" code. I apologise if this is a silly question, as Ihaven't really been able to figure out yet how dart_eval calls into "real dart".

    opened by maks 12
  • Making a Dart interpreter inside official dart repo?

    Making a Dart interpreter inside official dart repo?

    Hi, this project looks great and looks like a big amount of work! Excited to see it is still growing, and looking forward to seeing it in production :)

    If I understand correctly, this looks like a Dart interpreter. Is it possible that we directly contribute to the dartlang repo and create an interpreter there? They use C++ so I guess the interpretation may run faster since we can control more low-level things.

    Given that Dart already have JIT and AOT, adding an interpreted mode looks like something that will be accepted there.

    Bytedance maintain a fork of Dart that does interpretation, for a similar reason; one could use that (I think it's public?). (by hixie in https://github.com/flutter/flutter/issues/14330#issuecomment-1279610785

    So technically speaking, it should also be implementable.

    I really want to see hot-update become reality. That's why I am suggesting this. Because I am quite afraid many people will not choose an interpreter that is not in the dart repository.

    Another advantage is that, we can utilize all infra of dart sdk internals. There must be a lot of optimizations etc already there that we may even be able to reuse :)

    opened by fzyzcjy 8
  • Maybe post and discuss on with community?

    Maybe post and discuss on with community?

    If this is to be the solution of flutter hot update, IMHO discussing with more smart people may be helpful. For example, there may be serious problems that we both have not realized which completely blocks the package - good to know it earlier than later.

    In addition, since Hot Update is the most upvoted issue in Flutter repo, I guess there will be some more people interested in this package and have the open-source spirits. Then, this package may have some PR and develop faster, i.e. gain hot-update sooner.

    Possible locations that I come up with (probably also other places that I do no know):

    • reddit
    • dart github repo / flutter dart discord channel - maybe dart team will provide some insights
    opened by fzyzcjy 7
  • Can I build a project that on my device?

    Can I build a project that on my device?

    Hey friend, I want to create an IDE to edit my dart project. Use the created IDE I can work on the train, but on the documents I can't find a way to build a dart project.

    Maybe I will copy all my codes to a string and complete it, is there some easier way to do it?

    opened by normidar 5
  • Support for arrow function entrypoint in eval()

    Support for arrow function entrypoint in eval()

    We can't have an expression body because it throws a compile error, like

    String main() => 'Hello World!';
    

    It only accepts a block body, like

    String main() {
      return 'Hello World!';
    }
    

    Code

    import 'package:dart_eval/dart_eval.dart';
    
    void main() {
      print(
        eval('''
          String main() => 'Hello World!';
        ''')
      );
    }
    

    Error

    Click to expand
    Unhandled exception:
    CompileError: Unknown statement type FunctionDeclarationStatementImpl at unknown (file package:default/main.dart)
    #0      Compiler.compileSources
    package:dart_eval/…/compiler/compiler.dart:344
    #1      Compiler.compile
    package:dart_eval/…/compiler/compiler.dart:131
    #2      eval
    package:dart_eval/…/eval/eval.dart:35
    #3      main
    test\temp.dart:5
    #4      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:297:19)
    
    #5      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:192:12)
    Exited (255)
    
    opened by DrafaKiller 3
  • Error using example from Readme

    Error using example from Readme

    Hi, the following code is taken from README.md. It will result in list of Errors using ^0.4.1.

      test('dart_eval', () {
        final program = '''
          class Cat {
            Cat(this.name);
            final String name;
            String speak() {
              return name;
            }
          }
          String main() {
            final cat = Cat('Fluffy');
            return cat.speak();
          }
      ''';
    
        print(eval(program, function: 'main')); // -> 'F
      });
    
    : Error: The getter 'name2' isn't defined for the class 'NamedCompilationUnitMember'.
    ../…/compiler/compiler.dart:403
    - 'NamedCompilationUnitMember' is from 'package:analyzer/dart/ast/ast.dart' ('../../../../flutter_sdk/.pub-cache/hosted/pub.dartlang.org/analyzer-4.2.0/lib/dart/ast/ast.dart').
    package:analyzer/…/ast/ast.dart:1
    Try correcting the name to the name of an existing getter, or defining a getter or field named 'name2'.
          final name = declaration.name2.value() as String;
                                   ^^^^^
    : Error: The getter 'name2' isn't defined for the class 'MethodDeclaration'.
    ../…/compiler/compiler.dart:416
    - 'MethodDeclaration' is from 'package:analyzer/dart/ast/ast.dart' ('../../../../flutter_sdk/.pub-cache/hosted/pub.dartlang.org/analyzer-4.2.0/lib/dart/ast/ast.dart').
    package:analyzer/…/ast/ast.dart:1
    Try correcting the name to the name of an existing getter, or defining a getter or field named 'name2'.
                final mName = member.name2.value() as String;
    
                                     ^^^^^
    : Error: The getter 'name2' isn't defined for the class 'ClassDeclaration'.
    ../…/compiler/compiler.dart:432
    - 'ClassDeclaration' is from 'package:analyzer/dart/ast/ast.dart' ('../../../../flutter_sdk/.pub-cache/hosted/pub.dartlang.org/analyzer-4.2.0/lib/dart/ast/ast.dart').
    package:analyzer/…/ast/ast.dart:1
    Try correcting the name to the name of an existing getter, or defining a getter or field named 'name2'.
                    final name = (declaration.name2.value() as String) + '.' + (field.name2.value() as String);
    
    
    opened by smiLLe 3
  • Cannot open file, path = 'out.dbc' (OS Error: Read-only file system, errno = 30)

    Cannot open file, path = 'out.dbc' (OS Error: Read-only file system, errno = 30)

    ════════════════════════════════════════════════════════════════════════════════

    ════════ Exception caught by gesture ═══════════════════════════════════════════ Cannot open file, path = 'out.dbc' (OS Error: Read-only file system, errno = 30) ════════════════════════════════════════════════════════════════════════════════

    opened by hellokai55 3
  • Security Implications?

    Security Implications?

    Assuming that I download untrusted code to my Windows application written in Flutter, compile the code using dart_eval and execute it, can it do anything problematic, like access the file system or the network?

    As far as I can tell, dart:io is not exposed, but is there anything else in there?

    opened by anlumo 2
  • Comparison between this and JavaScript or Wasm based solutions?

    Comparison between this and JavaScript or Wasm based solutions?

    (I am not against you! Just because if we are going to make it the solution, we have to think about all possible opposite voices by ourselves.)

    So, what if someone (or us?) implements hot update via:

    1. JavaScript?
    2. Wasm?

    (Hixie also mentioned these in 14330)

    My non-mature answer:

    • JS/Wasm have a much faster runtime (compared with dart_eval)
    • But the bridge may be slow (or slower? or faster?)
    • If bridge takes most of the time (we must have data to support this), then they do not have overall advantage

    P.S. For some existing hot update solutions, here is a comparison article: https://juejin.cn/post/7033708048321347615 (Not in English, may need a translator...)

    opened by fzyzcjy 2
  • some other dart syntax goodness

    some other dart syntax goodness

    It looks like the ?? operator and the the ? : conditional syntax are not yet supported. I find these used all the time in flutter work, any chance these are on the roadmap ? Or even some pointers/advice on how to work thru adding these for a PR?

    opened by aktxyz 1
  • Support for Dart core types needed for String methods

    Support for Dart core types needed for String methods

    A number of core types are not yet supported in EvalTypes, but specifically the following are needed to be able to complete for all methods on String being done in #37 :

    • Pattern
    • Match
    • RegExp
    • Runes
    • Iterable

    Adding Iterable seems to be already covered by #9

    standard library 
    opened by maks 3
  • Potential optimization: Auto AOT when possible, and fallback to interpreted

    Potential optimization: Auto AOT when possible, and fallback to interpreted

    For example, say I release v1.0.0 app with 10 pages. Then in v1.0.1, I change the UI (i.e. HomePage.build), and a few logic (e.g. HomePage.handleMyButtonClick).

    Then, would be great to have all other pages (SecondPage, ThirdPage, ...) all in AOT, and also making all other untouched methods in HomePage still AOT, while only making HomePage.build and HomePage.handleMyButtonClick interpreted.

    This is because an app will not change most of things in a hot update.

    (Not sure whether I have mentioned this thought somewhere before)

    performance 
    opened by fzyzcjy 7
  • Potential optimization: Use integers instead of Strings to look up methods

    Potential optimization: Use integers instead of Strings to look up methods

    For example, strings here: https://github.com/ethanblake4/dart_eval/blob/c5795f1e284a46e44d2f9c30b73eebc3b2787fc3/lib/src/eval/shared/stdlib/core/map.dart#L20 "length" can become (index) 2 etc.

    This must be combined with the https://github.com/ethanblake4/dart_eval/issues/20 because it is so error-prone to manually assign numbers.

    If doing https://github.com/ethanblake4/dart_eval/issues/65, this may be of less priority, since lookup happens in secondary isolate instead of the main isolate, so may not be the bottleneck.

    performance 
    opened by fzyzcjy 6
Owner
Ethan
Ethan
A Dart build script that downloads the Protobuf compiler and Dart plugin to streamline .proto to .dart compilation.

A Dart build script that downloads the Protobuf compiler and Dart plugin to streamline .proto to .dart compilation.

Julien Scholz 10 Oct 26, 2022
Dart wrapper via dart:ffi for https://github.com/libusb/libusb

libusb Dart wrapper via dart:ffi for https://github.com/libusb/libusb Environment Windows(10) macOS Linux(Ubuntu 18.04 LTS) Usage Checkout example Fea

Woodemi Co., Ltd 28 Dec 20, 2022
Quiver is a set of utility libraries for Dart that makes using many Dart libraries easier and more convenient, or adds additional functionality.

Quiver is a set of utility libraries for Dart that makes using many Dart libraries easier and more convenient, or adds additional functionality.

Google 905 Jan 2, 2023
AOP for Flutter(Dart)

AspectD Salute to AspectJ. AspectD is an AOP(aspect oriented programming) framework for dart. Like other traditional aop framework, AspectD provides c

null 1k Jan 7, 2023
Environment specific config generator for Dart and Flutter applications during CI/CD builds

Environment Config Generator Environment specific config generator. Allows to specify env configuration during CI/CD build. Primarily created to simpl

Denis Beketsky 86 Dec 2, 2022
A Very Good Command Line Interface for Dart created by Very Good Ventures 🦄

Very Good CLI Developed with ?? by Very Good Ventures ?? A Very Good Command Line Interface for Dart. Installing $ dart pub global activate very_good_

Very Good Open Source 1.8k Jan 8, 2023
🚀The Flutter dart code generator from zeplin. ex) Container, Text, Color, TextStyle, ... - Save your time.

Flutter Gen Zeplin Extension ?? The Flutter dart code generator from zeplin. ex) Container, Text, Color, TextStyle, ... - Save your time. ⬇ 1.1k Getti

NAVER 49 Oct 12, 2022
Dart language version of Persian-Tools

Persian tools Persian Tools dart package which you can use in all platforms Features Adding ordinal suffixes Converting Persian words to number Conver

Persian Tools 54 Dec 29, 2022
A Dart package to web scraping data from websites easily and faster using less code lines.

Chaleno A flutter package to webscraping data from websites This package contains a set of high-level functions that make it easy to webscrap websites

António Nicolau 30 Dec 29, 2022
A pure dart package to apply useful rate limiting strategies on regular functions.

Rate limiting is a strategy for limiting an action. It puts a cap on how often someone can repeat an action within a certain timeframe. Using rate_limiter we made it easier than ever to apply these strategies on regular dart functions.

Stream 24 Dec 14, 2022
Dart phone number parser, based on libphonenumber and PhoneNumberKit.

Dart library for parsing phone numbers. Inspired by Google's libphonenumber and PhoneNumberKit for ios.

cedvdb 39 Dec 31, 2022
Basic Dart reverse shell code

dart_rs Basic Dart reverse shell based on this one by Potato-Industries. Pretty self explanatory. You’ll need Windows. I used a Windows 7 64-bit VM. F

null 21 Oct 2, 2022
Dart phone number parser, based on libphonenumber and PhoneNumberKit.

Phone Numbers Parser Dart library for parsing phone numbers. Inspired by Google's libphonenumber and PhoneNumberKit for ios. The advantage of this lib

cedvdb 39 Dec 31, 2022
Provides API to generate Dart source code

DartWriter DartWriter provides API to generate Dart source code. It can make your job easier while developing flutter/dart tools. You can also generat

Ahmet ÇELİK 11 Oct 24, 2022
Simple & Beautiful Note taking app written in dart with flutter UI toolkit.

Notes is a privacy oriented, Secure ,beautiful and fast application made in flutter, it supports various features like adding and saving notes. Hiding

null 26 Dec 30, 2022
A configurable pattern finder for static analysis written in Dart

mistdumper A configurable pattern finder for static analysis written in Dart. It is meant to retrieve offsets from file on disk without running them.

null 3 Sep 12, 2022
A Dart library to parse Portable Executable (PE) format

pefile A Dart library to parse Portable Executable (PE) format Usage A simple usage example: var pe = pefile.parse('C:\\Windows\\System32\\notepad.exe

null 4 Sep 12, 2022
Call OpenGL ES Api By Dart

Flutter GL Flutter GL can call OpenGL ES API with Dart Support iOS,Android,Web OpenGL ES API Now the api is similar to WebGL How to use Now this is on

zhaolei 163 Jan 5, 2023
Ecosistema de paquetes para Dart y Flutter desarrollados y mantenidos por la comunidad de Flutter Cuba relacionados con la tienda cubana de aplicaciones para dispositivos Android llamada Apklis.

Ecosistema de paquetes para Dart y Flutter desarrollados y mantenidos por la comunidad de Flutter Cuba relacionados con la tienda cubana de aplicacion

Flutter Cuba 12 Oct 24, 2022